Skip to content

Conversation

@kaushikmitr
Copy link
Contributor

This pull request introduces admission control logic to the SLOAwareRouter and improves robustness by adding nil request checks to lifecycle hooks. It also refactors the scoring and prediction logic for better separation of concerns and testability, and adds comprehensive unit tests for the new admission controller behavior.

Admission control and validation:

  • Implements the AdmitRequest method for SLOAwareRouter, enabling admission control based on SLO predictions. The method checks if a request has a valid pod according to SLO context and rejects requests otherwise. It also handles nil requests and missing SLO contexts gracefully.
  • Registers SLOAwareRouter as an AdmissionPlugin by updating the interface assertions.
  • Adds extensive unit tests for AdmitRequest, including edge cases (nil request, missing SLO context, valid/invalid pod scenarios, concurrent access, and error messages).

Robustness improvements:

  • Adds nil request checks with debug logging to all request lifecycle hooks (PreRequest, ResponseReceived, ResponseStreaming, ResponseComplete) to prevent panics and improve traceability.
    Prediction and scoring refactoring:

  • Refactors the generatePredictions method to remove the unused state parameter and simplifies prefix cache score retrieval by using the SLO context instead of recalculating.

  • Moves prefix cache score calculation into the Score method before prediction generation, ensuring scores are available in the SLO context for all candidate pods.

  • Updates the call to generatePredictions in Score to match the new signature, removing the state parameter.

  • Fixes logic for determining if all pods are invalid by considering SLOs

@netlify
Copy link

netlify bot commented Nov 23, 2025

Deploy Preview for gateway-api-inference-extension ready!

Name Link
🔨 Latest commit 1169246
🔍 Latest deploy log https://app.netlify.com/projects/gateway-api-inference-extension/deploys/692370d94b2dfe00083d9755
😎 Deploy Preview https://deploy-preview-1891--gateway-api-inference-extension.netlify.app
📱 Preview on mobile
Toggle QR Code...

QR Code

Use your smartphone camera to open QR code link.

To edit notification comments on pull requests, go to your Netlify project configuration.

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: kaushikmitr
Once this PR has been reviewed and has the lgtm label, please assign nirrozenbaum for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. labels Nov 23, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants