Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Frontend] Print out config when autotune fails #5117

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

htyu
Copy link
Collaborator

@htyu htyu commented Nov 11, 2024

Printing out config when autotune fails under TRITON_DEBUG. This will allow kernel developers to narrow down a problematic config and remove it to unblock themselves. This could also help compiler developers focus on debugging that config.

@htyu htyu requested a review from ptillet as a code owner November 11, 2024 16:58
@Jokeren
Copy link
Contributor

Jokeren commented Nov 12, 2024

Can you use TRITON_PRINT_AUTOTUNING?

@htyu
Copy link
Collaborator Author

htyu commented Nov 12, 2024

Can you use TRITON_PRINT_AUTOTUNING?

Sometimes TRITON_PRINT_AUTOTUNING isn't enough. For example, in the present of runtime failure or llvm failure that cannot captured by the compiler driver, the process simply quits, and users don't get a message about which autotune config went wrong.

@bertmaher
Copy link
Collaborator

Maybe instead of gating this functionality on TRITON_DEBUG it should be added to TRITON_PRINT_AUTOTUNING I feel like if you're using the print-autotuning flag, you're probably debugging something about autotuning, and it might be nicer to have a single flag for that instead of two, particularly since TRITON_DEBUG also gates some debugging features in codegen (device-side asserts and stuff)

@htyu
Copy link
Collaborator Author

htyu commented Nov 13, 2024

Maybe instead of gating this functionality on TRITON_DEBUG it should be added to TRITON_PRINT_AUTOTUNING I feel like if you're using the print-autotuning flag, you're probably debugging something about autotuning, and it might be nicer to have a single flag for that instead of two, particularly since TRITON_DEBUG also gates some debugging features in codegen (device-side asserts and stuff)

Sounds good. Swithed to using TRITON_PRINT_AUTOTUNING.

Copy link
Collaborator

@bertmaher bertmaher left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Lgtm! I’ll wait a bit to merge just in case anyone else wants to weigh in

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants