Option to serve external IPs

[johnbelamaric]

What do you all think about the idea of serving up the load balancer VIP instead of the clusterIP, at least if some option is set.

So, the config for the connection to the local API server would still serve up clusterIPs, but the configs for the other clusters would have some bit to tell it to serve up the load balancer VIP. Of course it doesn't help for pure NodePort or ClusterIP services, but would allow cross-cluster discovery for LB services.

This might be a bad idea. Another option is to somehow combine the k8s_external and kubernetai to do a similar thing, either directly or through zone transfer or something. Anyway, just a thought to get some discussion going.

[chrisohaver]

Yes, I think it's a good idea. I would prefer extending this plugin with an option over trying to do it with k8s_external (which seems complicated at first glance - re: fallthrough).

IMO, for services without a load balancer or external ip, we could use the cluster ip. IOW, the option would be something like prefer (external|internal) where prefer internal is the default.

prefer external: Serve load balancer/external IPs for a service if it has them instead of cluster ip (or endpoint ids for headless service)

prefer internal: Serve cluster IPs, or endpoint ips for headless services (e.g. act as we do now). This option is not really needed, since it is the default behavior.

In that vein, another useful option...
prefer endpoints: Could be an option to always serve endpoint ips, even for ClusterIP services. I think this makes good sense for cross cluster use... With this option set, a user would not have to create headless service "shadows" for every clusterip service. Maybe should be a separate option, since someone might want to use both external and endpoints... maybe allow both in one option: i.e. prefer external endpoints (but confusing sounding when you read it out).

All of this at least requires working out how to include additional options for kubernetai that are not in kubernetes. e.g. parse and scrape out kubernetai options before parsing with kubernetes. Which is doable.

[johnbelamaric]

interesting. not sure if prefer external as you describe it would be good for headless services. I guess maybe, since probably the wouldn't normally have any external IPs, so typically it would still serve the endpoint IPs.