Initial InferencePool Status for Inference Extension EPP Plugin

[danehans]

Description

Adds support for InferencePool status to the inference extension endpointpicker plugin.

Change Type

/kind new_feature

Changelog

Adds InferencePool status management to Inference Extension endpointpicker (EPP) Plugin.

Fixes #11306

[danehans]

Note that I implemented InferencePool status management in the plugin instead of the controllers package because I plan to remove the inference extension EPP deployer.

[shashankram]

I prefer errors delimited by a semicolon or \n (errors.Join). Why quote this?

[danehans]

I didn't use errors.Join since it creates a new line between errors, which does not render well in status messages. I'll update to remove the quoted error messages and separate each with a semicolon.

[shashankram]

You don't need this. You can use FetchHTTPRoutesBySelector and only set the Namespace in the selector

[danehans]

Then I need to plumb a krt.HandlerContext throughout the call chain.

[shashankram]

I am unfamiliar with krt.HandlerContext, but the pattern of passing it all the way through the callstack is used everywhere (Translate entrypoints). FetchHTTPRoutesBySelector uses a namespaced index, so lookups would be O(1) vs O(n) with ListHTTPRoutesInNamespace.

Is it a big change to make?

[danehans]

@shashankram I don't expect a large number of HTTPRoutes in the InferencePool's namespace. For the time being, I would like to stick with the current approach and implement this improvement in #11379.