-
Notifications
You must be signed in to change notification settings - Fork 168
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Evict requests if the client has disconnected #165
Comments
Hi @aniketmaurya, I've been exploring potential implementations for this feature Looking forward to your thoughts on it! 😊 |
sounds great @bhimrazy!! the only thing to take care of would be that we send the request to another process so Another idea could be is to check using socket connection but we can try that later. |
Thank you, @aniketmaurya, for the insights, and apologies for the delay in drafting the PR. |
🚀 Feature
If a client sends a request and is disconnected, LitServe still processes the request which is a waste of computation. We need to evict the request if the client is disconnected.
Motivation
Pitch
Alternatives
Additional context
The text was updated successfully, but these errors were encountered: