-
Notifications
You must be signed in to change notification settings - Fork 18
Update warmup log messages and comments #284
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: Travis Johnson <[email protected]>
👋 Hi! Thank you for contributing to vLLM support on Spyre.
Or this can be done with
Now you are good to go 🚀 |
I took the liberty of updating most of the warmup logging: using a |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM! Couple nits
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
lgtm, if possible address @prashantgupta24 nits..
Signed-off-by: Travis Johnson <[email protected]>
get in! |
### [SB] fix order of warmup print order changed in this this PR: #284 Signed-off-by: Yannick Schnider <[email protected]>
Description
This came out of a follow up to #270 to determine why an extra Prefill was necessary after using
warmup_mode
. I learned that the extra Prefill is required to deploy the compiled graph to the Spyre device. This PR does not change any functionality, but updates logging and documentation around warmup to make this clearer.