Support multiple "ingressClass" which point to different "ingressSelector"

[mitre-dleung]

(This is used to request new product features, please visit https://discuss.istio.io for questions on using Istio)

Have a need to support multiple istio ingressgateway deployments with different LoadBalancer frontend IPs due to performance and scalability concerns. Currently unable to use cert-manager for certificate issuance with multiple istio ingressgateways deployments.

Currently, in the istio MeshConfig; only 1 ingressSelector and ingressClass can be configured. This results in only 1 istio ingressgateway deployment to be selectable as the ingress controller for the IngressClass. This prevents being able to configure multiple Cert-manager ClusterIssuers which specify an IngressClass in the ingress template to point to each of the different ingressgateway deployments.

So if the IngressSelector is configured to istio ingressgateway deployment A, any certificate challenges for domains served by ingressgateway deployment B will get sent to the LoadBalancer frontend IP for A, which will fail to resolve the domain.

Request: support multiple ingressClass and ingressSelector to support the multiple istio ingressgateway deployment use case.

Describe alternatives you've considered
Tried putting cert-manager inside the istio service mesh so I could use istio egress routes to re-route all cert-manager cert issuance connections to their respective istio ingressgateway deployment based on FQDN/URI, but the cert-manager controller component is incompatible with istio-proxy and fails to talk to its webhook component.

Affected product area (please put an X in all that apply)

[x] Configuration Infrastructure
[x] Docs
[ ] Installation
[ ] Networking
[x] Performance and Scalability
[ ] Policies and Telemetry
[ ] Security
[ ] Test and Release
[ ] User Experience

Additional context

[hzxuzhonghu]

could you use the ingressSelector to match multi deployments? Ingress is not on the long term support path. Change to gateway api?

[mitre-dleung]

could you use the ingressSelector to match multi deployments? Ingress is not on the long term support path. Change to gateway api?

It seems ingressSelector can only take one value for istio: <INGRESS_SELECTOR>, so only 1 label can be matched. The different ingressgateway deployments can't have the same istio: <INGRESS_SELECTOR> label because then the gateways can't be selected separately for different routes (which defeats the purpose of the separate ingressgateways for handling different sets of connections).

I'll look into gateway api. We had tried to install it a couple years ago but couldn't get it working at the time (alpha).