Update SLO aware router with Admit Request Plugin #1891
+188
−5
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This pull request introduces admission control logic to the
SLOAwareRouterand improves robustness by adding nil request checks to lifecycle hooks. It also refactors the scoring and prediction logic for better separation of concerns and testability, and adds comprehensive unit tests for the new admission controller behavior.Admission control and validation:
AdmitRequestmethod forSLOAwareRouter, enabling admission control based on SLO predictions. The method checks if a request has a valid pod according to SLO context and rejects requests otherwise. It also handles nil requests and missing SLO contexts gracefully.SLOAwareRouteras anAdmissionPluginby updating the interface assertions.AdmitRequest, including edge cases (nil request, missing SLO context, valid/invalid pod scenarios, concurrent access, and error messages).Robustness improvements:
Adds nil request checks with debug logging to all request lifecycle hooks (
PreRequest,ResponseReceived,ResponseStreaming,ResponseComplete) to prevent panics and improve traceability.Prediction and scoring refactoring:
Refactors the
generatePredictionsmethod to remove the unusedstateparameter and simplifies prefix cache score retrieval by using the SLO context instead of recalculating.Moves prefix cache score calculation into the
Scoremethod before prediction generation, ensuring scores are available in the SLO context for all candidate pods.Updates the call to
generatePredictionsinScoreto match the new signature, removing thestateparameter.Fixes logic for determining if all pods are invalid by considering SLOs