feat(source/service): watch headless endpoints

[tufitko]

Description

Call event handler for service source when headless service's endpoints change.

Fixes #4113

Checklist

• Unit tests updated
• End user documentation updated

[k8s-ci-robot]

Welcome @tufitko!

It looks like this is your first PR to kubernetes-sigs/external-dns 🎉. Please refer to our pull request process documentation to help your PR have a smooth ride to approval.

You will be prompted by a bot to use commands during the review process. Do not be afraid to follow the prompts! It is okay to experiment. Here is the bot commands documentation.

You can also check if kubernetes-sigs/external-dns has its own contribution guidelines.

You may want to refer to our testing guide if you run into trouble with your tests not passing.

If you are having difficulty getting your pull request seen, please follow the recommended escalation practices. Also, for tips and tricks in the contribution process you may want to read the Kubernetes contributor cheat sheet. We want to make sure your contribution gets all the attention it needs!

Thank you, and welcome to Kubernetes. 😃

[k8s-ci-robot]

Hi @tufitko. Thanks for your PR.

I'm waiting for a kubernetes-sigs member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign mloiseleur for approval. For more information see the Kubernetes Code Review Process.

The full list of commands accepted by this bot can be found here.

Needs approval from an approver in each of these files:

• OWNERS

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[mloiseleur]

Thanks. This PR looks good.
/ok-to-test

[mloiseleur]

/lgtm

[szuecs]

I would rather like to see an opt-in approach rather than having this added.

[tufitko]

hmm, I can add label selector to a shared informer factory, but I need to create another one, something like this:

	headlessLabelSelector, err := labels.NewRequirement(v1.IsHeadlessService, selection.Exists, nil)
	if err != nil {
		return nil, err
	}

	headlessEndpointInformerFactory := kubeinformers.NewSharedInformerFactoryWithOptions(kubeClient, 0,
		kubeinformers.WithNamespace(namespace),
		kubeinformers.WithTweakListOptions(func(options *metav1.ListOptions) {
			options.LabelSelector = headlessLabelSelector.String()
		}),
	)
	
	endpointsInformer := headlessEndpointInformerFactory.Core().V1().Endpoints()
	
	// duplicate Start and waitForCacheSync funcs for this factory

Is that OK?

[tufitko]

so, do I need to do something?

[tufitko]

@szuecs

[orgads]

@szuecs @mloiseleur Can you please reply?

[tufitko]

@mloiseleur Are you sure it’s not enabled by default? It would be strange behavior if you don’t use the filter but still don’t get everything. Additionally, this filter seems to work differently compared to the others.

[mloiseleur]

On v0.15.0, it's all:

  --service-type-filter=SERVICE-TYPE-FILTER ...  
         The service types to take care about (default: all, 
               expected: ClusterIP, NodePort, LoadBalancer or ExternalName)

I think you're right: it works quite differently from the other service types. It would be quite unexpected to enable/disable an informer with this filter.

So I guess it should be opt-in with a new option on --source (something like headless-service for instance)

[tufitko]

hmm, I think it better to add explicit type in --service-type-filter, enabled by default.
Anyway, it's breaking changes.

With a new source, 2 sources are triggered when creating/updating/deleting services (headless or not) because there isn't any filter in the event handler. Is that OK?

With a new service type (enabled by default), the source can be triggered more often.
For users, who use external-dns for headless services, this change would be good (allows increasing sync interval)
For users, who don't have headless services in a cluster, nothing is changed
Bad case, when a user has a headless service, but doesn't use external-dns for managing records

[Raffo]

I think the last proposal from @tufitko makes sense to me. I think this is more of a bug fix than something we need to carefully feature flag and it's gonna be in the new release only anyway. wdyt @mloiseleur ?

[mloiseleur]

@Raffo that works for me.

@tufitko any questions left ?

[ivankatliarchuk]

Hi @tufitko

Are you still interesting in this feature to be merged? I appreciate is being a while.

Just to make it clear, the headler service is this one? Just to make sure we're talking about the same thing...

So we have dozens of headless services, mainly grpc client/service applications. We have client load balancers, they root traffic to in-cluster ips with cluster local, and we have ingresses if grpc calls happens cross-clusters or networks. There are as well headless services that route traffic to S3 or similar storages and etc.

Here is the very high level diagram

What come to mind, potential pain points I see here

• Given the high number of pods behind headless services and our reliance on spot instances, which involve frequent node migrations, the current proposal may face significant challenges. With DNS providers often implementing rate limits on their APIs, I'm concerned that the system could easily exceed those limits, leading to instability and service disruptions/unavailability.

Will this change affect all the headless services, or only specific services annotate with which annotation?

[ivankatliarchuk]

/assign

[k8s-ci-robot]

PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.