Secondary/Additional Domains

[jalim]

I was interested in hosting a service using an secondary/additional domain name while still utilising the single Cloudflared ingress. Seeing as there wasn't documentation for this case I'll try to document what I modified to make it all work. For reference, both primary and secondary domains were both using cloudflare and my cloudflare token had permissions for both zones.

In order for cert-manager to order new certificates I added the secondary domain to the issuer:

apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: letsencrypt-production
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: "${SECRET_ACME_EMAIL}"
    privateKeySecretRef:
      name: letsencrypt-production
    solvers:
      - dns01:
          cloudflare:
            apiTokenSecretRef:
              name: cert-manager-secret
              key: api-token
        selector:
          dnsZones:
            - "${SECRET_DOMAIN}"
            - "${SECONDARY_DOMAIN"

and created a new certificate request in the nginx config:

---
apiVersion: cert-manager.io/v1
kind: Certificate
metadata:
  name: "${SECONDARY_DOMAIN/./-}-production"
  namespace: networking
spec:
  secretName: "${SECONDARY_DOMAIN/./-}-production-tls"
  issuerRef:
    name: letsencrypt-production
    kind: ClusterIssuer
  commonName: "${SECONDARY_DOMAIN}"
  dnsNames:
    - "${SECONDARY_DOMAIN"
    - "*${SECONDARY_DOMAIN"

In order for external-dns to add the correct cname records I added to the domain filter in the helmrelease as such:

    ...
    txtOwnerId: default
    domainFilters: ["${SECRET_DOMAIN}","${SECONDARY_DOMAIN"]
    serviceMonitor:
    ...

Then I added ingress rules to the cloudflared config as such:

---
originRequest:
  http2Origin: true

# https://developers.cloudflare.com/cloudflare-one/connections/connect-apps/configuration/ingress
ingress:
  - hostname: "${SECRET_DOMAIN}"
    service: https://ingress-nginx-controller.networking.svc.cluster.local:443
    originRequest:
      originServerName: "ingress.${SECRET_DOMAIN}"
  - hostname: "*.${SECRET_DOMAIN}"
    service: https://ingress-nginx-controller.networking.svc.cluster.local:443
    originRequest:
      originServerName: "ingress.${SECRET_DOMAIN}"
  - hostname: "${SECONDARY_DOMAIN}"
    service: https://ingress-nginx-controller.networking.svc.cluster.local:443
    originRequest:
      originServerName: "ingress.${SECRET_DOMAIN}"
  - hostname: "*.${SECONDARY_DOMAIN}"
    service: https://ingress-nginx-controller.networking.svc.cluster.local:443
    originRequest:
      originServerName: "ingress.${SECRET_DOMAIN}"
  - service: http_status:404

To test I added the following to echo-server HR:

    ingress:
      main:
        enabled: true
        ingressClassName: nginx
        annotations:
          external-dns.alpha.kubernetes.io/target: "ingress.${SECRET_DOMAIN}"
          hajimari.io/icon: video-input-antenna
        hosts:
          - host: &host "{{ .Release.Name }}.${SECRET_DOMAIN}"
            paths:
              - path: /
                pathType: Prefix
          - host: &host1 "{{ .Release.Name }}.${SECONDARY_DOMAIN}"
            paths:
              - path: /
                pathType: Prefix
        tls:
          - hosts:
              - *host
              - *host1

All seems to work smoothly and as expected.

[h3mmy]

Nice! I've run been running multiple domains but I'm not using external-dns yet so this will be good to reference.

Something you may want to note is to ensure adding all domains in your content-security-policy frame-ancestors and adjusting accessControlAllow lists appropriately. Not everyone uses security headers, so I'm not sure how widely applicable this would be.

[erenfro]

And now, k8s-gateway, for DNS resolution?

[erenfro]

And also, the echo-server seems to use the older nginx classes, rather than the className: external approach of the new template format.
I'm hitting the case where, specifically echo-server, is always fishing out the default domain cert.