Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: allow pod IPs even for non-hostNetwork pods #3174

Closed
wants to merge 2 commits into from
Closed

feat: allow pod IPs even for non-hostNetwork pods #3174

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
365c0b0
feat: allow pod IPs even for non-hostNetwork pods
peterhoneder Dec 21, 2023
fbd4bc5
fixed review comments
peterhoneder Jan 22, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion docs/annotations/annotations.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,8 @@ Specifies the domain for the resource's DNS records.
Multiple hostnames can be specified through a comma-separated list, e.g.
`svc.mydomain1.com,svc.mydomain2.com`.

For `Pods`, uses the `Pod`'s `Status.PodIP`, unless they are `hostNetwork: true` in which case the NodeExternalIP is used for IPv4 and NodeInternalIP for IPv6.

## external-dns.alpha.kubernetes.io/ingress-hostname-source

Specifies where to get the domain for an `Ingress` resource.
Expand All @@ -80,7 +82,7 @@ Specifies the domain for the resource's DNS records that are for use from intern

For `Services` of type `LoadBalancer`, uses the `Service`'s `ClusterIP`.

For `Pods`, uses the `Pod`'s `Status.PodIP`.
For `Pods`, uses the `Pod`'s `Status.PodIP`, unless they are `hostNetwork: true` in which case the NodeExternalIP is used for IPv4 and NodeInternalIP for IPv6.

## external-dns.alpha.kubernetes.io/target

Expand Down
6 changes: 2 additions & 4 deletions docs/tutorials/kops-dns-controller.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,11 +21,9 @@ The DNS record mappings try to "do the right thing", but what this means is diff

### Pods

For the external annotation, ExternalDNS will map a HostNetwork=true Pod to the external IPs of the Node.
For the external annotation, ExternalDNS will map a Pod to the external IPs of the Node.

For the internal annotation, ExternalDNS will map a HostNetwork=true Pod to the internal IPs of the Node.

ExternalDNS ignore Pods that are not HostNetwork=true
For the internal annotation, ExternalDNS will map a Pod to the internal IPs of the Node.

Annotations added to Pods will always result in an A record being created.

Expand Down
15 changes: 9 additions & 6 deletions source/pod.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,7 +21,6 @@ import (

"sigs.k8s.io/external-dns/endpoint"

log "github.com/sirupsen/logrus"
corev1 "k8s.io/api/core/v1"
"k8s.io/apimachinery/pkg/labels"
kubeinformers "k8s.io/client-go/informers"
Expand Down Expand Up @@ -84,13 +83,9 @@ func (ps *podSource) Endpoints(ctx context.Context) ([]*endpoint.Endpoint, error

endpointMap := make(map[endpoint.EndpointKey][]string)
for _, pod := range pods {
if !pod.Spec.HostNetwork {
log.Debugf("skipping pod %s. hostNetwork=false", pod.Name)
continue
}

targets := getTargetsFromTargetAnnotation(pod.Annotations)

// accept internal hostname annotations that point to the pod's IP
if domainAnnotation, ok := pod.Annotations[internalHostnameAnnotationKey]; ok {
domainList := splitHostnameAnnotation(domainAnnotation)
for _, domain := range domainList {
Expand All @@ -104,6 +99,7 @@ func (ps *podSource) Endpoints(ctx context.Context) ([]*endpoint.Endpoint, error
}
}

// accept internal hostname annotations that point to the pod's IP (or IPv6 NodeInternalIP)
if domainAnnotation, ok := pod.Annotations[hostnameAnnotationKey]; ok {
domainList := splitHostnameAnnotation(domainAnnotation)
for _, domain := range domainList {
Expand Down Expand Up @@ -162,5 +158,12 @@ func addToEndpointMap(endpointMap map[endpoint.EndpointKey][]string, domain stri
if _, ok := endpointMap[key]; !ok {
endpointMap[key] = []string{}
}

// check that only unique addresses are added
for _, existingAddress := range endpointMap[key] {
if existingAddress == address {
return
}
}
endpointMap[key] = append(endpointMap[key], address)
}
104 changes: 94 additions & 10 deletions source/pod_test.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -116,7 +116,7 @@ func TestPodSource(t *testing.T) {
"kops-dns-controller",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1", "54.10.11.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1", "10.0.1.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1", "10.0.1.2", "10.0.1.3"}, RecordType: endpoint.RecordTypeA},
},
false,
[]*corev1.Node{
Expand Down Expand Up @@ -178,6 +178,26 @@ func TestPodSource(t *testing.T) {
PodIP: "10.0.1.2",
},
},
// non-HostNetwork pod
// - internal hostname annotation will use the PodIP
// - external hostname annotation will use the NodeExternalIP
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod3",
Namespace: "kube-system",
Annotations: map[string]string{
kopsDNSControllerInternalHostnameAnnotationKey: "internal.a.foo.example.org",
kopsDNSControllerHostnameAnnotationKey: "a.foo.example.org",
},
},
Spec: corev1.PodSpec{
HostNetwork: false,
NodeName: "my-node2",
},
Status: corev1.PodStatus{
PodIP: "10.0.1.3",
},
},
},
},
{
Expand All @@ -186,7 +206,7 @@ func TestPodSource(t *testing.T) {
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2", "2001:DB8::3"}, RecordType: endpoint.RecordTypeAAAA},
},
false,
[]*corev1.Node{
Expand Down Expand Up @@ -246,6 +266,26 @@ func TestPodSource(t *testing.T) {
PodIP: "2001:DB8::2",
},
},
// this pod's hostname annotations (both internal and not) must not be ignored even though it's not in the host network
// - internal hostname annotation uses the PodIP
// - external hostname annotation uses the NodeInternalIP
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod3",
Namespace: "kube-system",
Annotations: map[string]string{
internalHostnameAnnotationKey: "internal.a.foo.example.org",
hostnameAnnotationKey: "a.foo.example.org",
},
},
Spec: corev1.PodSpec{
HostNetwork: false,
NodeName: "my-node2",
},
Status: corev1.PodStatus{
PodIP: "2001:DB8::3",
},
},
},
},
{
Expand All @@ -254,7 +294,7 @@ func TestPodSource(t *testing.T) {
"kops-dns-controller",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2"}, RecordType: endpoint.RecordTypeAAAA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"2001:DB8::1", "2001:DB8::2", "2001:DB8::3"}, RecordType: endpoint.RecordTypeAAAA},
},
false,
[]*corev1.Node{
Expand Down Expand Up @@ -314,15 +354,35 @@ func TestPodSource(t *testing.T) {
PodIP: "2001:DB8::2",
},
},
// this pod's hostname annotations (both internal and not) must not be ignored even though it's not in the host network
// - internal hostname annotation uses the PodIP
// - external hostname annotation uses the NodeInternalIP
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod3",
Namespace: "kube-system",
Annotations: map[string]string{
kopsDNSControllerInternalHostnameAnnotationKey: "internal.a.foo.example.org",
kopsDNSControllerHostnameAnnotationKey: "a.foo.example.org",
},
},
Spec: corev1.PodSpec{
HostNetwork: false,
NodeName: "my-node2",
},
Status: corev1.PodStatus{
PodIP: "2001:DB8::3",
},
},
},
},
{
"create records based on pod's target annotation",
"",
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"208.1.2.1", "208.1.2.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"208.1.2.1", "208.1.2.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"208.1.2.1", "208.1.2.2", "208.1.2.3"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"208.1.2.1", "208.1.2.2", "208.1.2.3"}, RecordType: endpoint.RecordTypeA},
},
false,
[]*corev1.Node{
Expand Down Expand Up @@ -386,6 +446,25 @@ func TestPodSource(t *testing.T) {
PodIP: "10.0.1.2",
},
},
// test non-HostNetwork pod
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod3",
Namespace: "kube-system",
Annotations: map[string]string{
internalHostnameAnnotationKey: "internal.a.foo.example.org",
hostnameAnnotationKey: "a.foo.example.org",
targetAnnotationKey: "208.1.2.3",
},
},
Spec: corev1.PodSpec{
HostNetwork: false,
NodeName: "my-node2",
},
Status: corev1.PodStatus{
PodIP: "10.0.1.3",
},
},
},
},
{
Expand Down Expand Up @@ -459,12 +538,16 @@ func TestPodSource(t *testing.T) {
},
},
{
"pods with hostNetwore=false should be ignored",
"hostname annotations of pods with hostNetwork=false should still be added as valid records",
"",
"",
[]*endpoint.Endpoint{
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1"}, RecordType: endpoint.RecordTypeA},
{DNSName: "a.foo.example.org", Targets: endpoint.Targets{"54.10.11.1", "54.10.11.2"}, RecordType: endpoint.RecordTypeA},
{DNSName: "internal.a.foo.example.org", Targets: endpoint.Targets{"10.0.1.1", "100.0.1.2"}, RecordType: endpoint.RecordTypeA},
Copy link
Contributor

@szuecs szuecs May 2, 2023 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The test name seems to be wrong.

And I wonder if you can increase test cases.

// this annotation is part of a comma separated hostname list in the annotation
{DNSName: "internal.b.foo.example.org", Targets: endpoint.Targets{"10.0.1.1"}, RecordType: endpoint.RecordTypeA},
// this annotation is part of a comma separated hostname list in the annotation
{DNSName: "internal.c.foo.example.org", Targets: endpoint.Targets{"100.0.1.2"}, RecordType: endpoint.RecordTypeA},
},
false,
[]*corev1.Node{
Expand All @@ -491,13 +574,14 @@ func TestPodSource(t *testing.T) {
},
},
},
// a mixed set of a host network pod and non host network pod should include both pod IPs of the internal hostname annotation in the endpoint
[]*corev1.Pod{
{
ObjectMeta: metav1.ObjectMeta{
Name: "my-pod1",
Namespace: "kube-system",
Annotations: map[string]string{
internalHostnameAnnotationKey: "internal.a.foo.example.org",
internalHostnameAnnotationKey: "internal.a.foo.example.org,internal.b.foo.example.org",
hostnameAnnotationKey: "a.foo.example.org",
},
},
Expand All @@ -514,7 +598,7 @@ func TestPodSource(t *testing.T) {
Name: "my-pod2",
Namespace: "kube-system",
Annotations: map[string]string{
internalHostnameAnnotationKey: "internal.a.foo.example.org",
internalHostnameAnnotationKey: "internal.a.foo.example.org,internal.c.foo.example.org",
hostnameAnnotationKey: "a.foo.example.org",
},
},
Expand Down
Loading