Inconsistent behavior for replacements directive

[emirot]

What happened?

I really don't understand the behavior of replacements and it looks like a bug to me.
Let me show you how to reproduce

Context: I would like in my ingress resources to add a generated secret in nginx.ingress.kubernetes.io/auth-tls-secret: mynamespace/my-genreated-secret-123ada15b3 I used Vars to do that and worked, trying to convert to replacements to stay compatible with newer Kustomize version but it gives me unexpected results.

Example 1

Resources

Ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: private-http-api-ingress
  annotations:
    ns: "my-ns"
    nolan: "repalce-me"
    nginx.ingress.kubernetes.io/auth-tls-secret: "myns/"
spec:
  tls:
    - hosts:
        - test.com

resources:
 - ingress.yaml

secretGenerator:
- name: my-generated-secret
  type: Opaque
  literals:
  - username=myuser
  - password=mysecurepassword

# E.g 1 will replace result in nolan: my-generated-secret
# Same for kustomze 4.5.7 and 5.6.0
replacements:
  - source:
      kind: Secret
      name: my-generated-secret
      version: v1
    targets:
      - select:
          kind: Ingress
          group: networking.k8s.io
          version: v1
        fieldPaths:
          - metadata.annotations.nolan

Result

apiVersion: v1
data:
  password: bXlzZWN1cmVwYXNzd29yZA==
  username: bXl1c2Vy
kind: Secret
metadata:
  name: my-generated-secret-h2dm9f76k2
type: Opaque
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-secret: myns/
    nolan: my-generated-secret
....

=> It does not give me the generated name my-generated-secret-h2dm9f76k2

Example 2

Ingress.yaml

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: private-http-api-ingress
  annotations:
    ns: "my-ns"
    nolan: "repalce-me"
    nginx.ingress.kubernetes.io/auth-tls-secret: "myns/"
spec:
  tls:
    - hosts:
        - test.com

Kustomization.yaml

replacements:
  - source:
      kind: Secret
      name: my-generated-secret
      version: v1
    targets:
      - select:
          kind: Ingress
          group: networking.k8s.io
          version: v1
        fieldPaths:
          - metadata.annotations.[nginx.ingress.kubernetes.io/auth-tls-secret]

Result

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-secret: my-generated-secret-h2dm9f76k2 
    nolan: repalce-me
    ns: my-ns
  name: private-http-api-ingress

=> It replaces to the generated name in that case but not if I use metadata.annotations.nolan I m even more confused now

Example 3:

Ingress

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: private-http-api-ingress
  annotations:
    ns: "my-ns"
    nolan: "repalce-me"
    nginx.ingress.kubernetes.io/auth-tls-secret: "myns/"
spec:
  tls:
    - hosts:
        - test.com

Kustomization.yaml

replacements:
  - source:
      kind: Secret
      name: my-generated-secret
      version: v1
    targets:
      - select:
          kind: Ingress
          group: networking.k8s.io
          version: v1
        fieldPaths:
          - metadata.annotations.[nginx.ingress.kubernetes.io/auth-tls-secret]
        options:
          delimiter: "/" # I m adding `myns/` in the nginx annotation
          index: 1
          create: false

Result

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-tls-secret: myns/my-generated-secret
    nolan: repalce-me
    ns: my-ns
  name: private-http-api-ingress

=> My adding the option it removed the hash part and put myns/my-generated-secret

What did you expect to happen?

That the generated name will be used consistently my-generated-secret-h2dm9f76k2 but instead those three examples will give 3 different outputs

Kustomize version

5.6.0 also tested with 4.5.7 giving me same results

Operating system

MacOS

[k8s-ci-robot]

This issue is currently awaiting triage.

SIG CLI takes a lead on issue triage for this repo, but any Kubernetes member can accept issues by applying the triage/accepted label.

The triage/accepted label can be added by org members by writing /triage accepted in a comment.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[sda399]

names of resources are tracked with eventual hash suffixes in paths that are declared to hold name references.
.../auth-tls-secret is part of the default list of paths that hold name references.
if you need new paths to hold name references, you need to declare them.

when you prefix with 'myns/' my guess is that the engine cannot track it anymore. you renamed it and is not recognized as a reference to an existing resource.

#! /bin/sh

base=base
[ ! -d $base ] || rm -r $base
mkdir -p $base

cat <<EOF > $base/ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: myingress
  annotations:
    ns: "my-ns"
    nginx.ingress.kubernetes.io/auth-tls-secret: "ns/"
spec:
  tls:
    - hosts:
        - test.com
EOF

cat <<EOF > $base/kustomization.yaml
resources:
 - ingress.yaml

secretGenerator:
- name: sec
  type: Opaque
  literals:
  - username=myuser
  - password=mysecurepassword

replacements:
  - source:
      kind: Secret
      name: sec
    targets:
      - select:
          kind: Ingress
        fieldPaths:
          - metadata.annotations.[nginx.ingress.kubernetes.io/auth-tls-secret]
          - metadata.annotations.[nginx.ingress.kubernetes.io/auth-secret]
          - metadata.annotations.nolan
          - spec.newnameref
        options:
          create: true

configurations:
  - namerefs.yaml
EOF

cat <<EOF > $base/namerefs.yaml
nameReference:
  - kind: Secret
    version: v1
    fieldSpecs:
      - path: spec/newnameref
        kind: Ingress
      - path: metadata/annotations/nolan
        kind: Ingress
EOF

cat <<EOF > expected.yaml
apiVersion: v1
data:
  password: bXlzZWN1cmVwYXNzd29yZA==
  username: bXl1c2Vy
kind: Secret
metadata:
  name: sec-h2dm9f76k2
type: Opaque
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  annotations:
    nginx.ingress.kubernetes.io/auth-secret: sec-h2dm9f76k2
    nginx.ingress.kubernetes.io/auth-tls-secret: sec-h2dm9f76k2
    nolan: sec-h2dm9f76k2
    ns: my-ns
  name: myingress
spec:
  newnameref: sec-h2dm9f76k2
  tls:
  - hosts:
    - test.com
EOF

#---
kustomize build --stack-trace $base > result.yaml
printf "%-64s%s\n" expected.yaml result.yaml
printf "%130s\n" " " | tr ' ' '-'
diff -W150 -y expected.yaml result.yaml

[emirot]

Thanks @sda399 in my case it would have been handy if secrets could use a prefix.

For nginx-ingresss-controller for instance, the auth-tis-secret can be passed in the annotation.
With vars in previous versions of kustomize it was easy, I could do "my-namespace/$(MY_VAR)"
but with newer version of kustomize it is not the case anymore.
From my example and yours it looks like I can have a prefix or suffix not both.

[sda399]

with vars you have hash suffix?

You can still get rid of hash suffix when generating your secret

[emirot]

With VARS yes it keeps the hash-suffix.

Yes, I know, I can disable it by doing

generatorOptions:
  disableNameSuffixHash: true

but by doing so I won't have my deployments redeployed when a secret change. ( the purpose of those hash suffix)

I will need to use something like https://github.com/stakater/Reloader to keep the same behavior.

[sda399]

this seems to be known issue: #4034

I would use vars if it works, it is not removed yet :)