Substitutions for setters

[squaremo]

Sometimes the image ref or tag (or digest) is part of a longer string in a field value. For example, in a container argument:

kind: Pod
spec:
  containers:
  - name: hello
    image: helloworld
    args:
    - --tag-argument=1.9.0 # <-- here

Using a setter won't work, since it'll replace the whole value. What's needed is a way to give a template for the whole value.

The existing solution to this in kyaml is "substitutions": you define a substitution in your schema file, which has a template with references to setter names; then, you name the substitution alongside the field. This is tricky in image-update-automation for a couple of reasons:

• there's no schema file in which to put substitutions; possibly they could go in the automation, but this is distant to the point at which it is applied, so seems like the wrong place;
• with simple setters, any that aren't defined (because they aren't in the namespace, or don't exist) will be ignored. How do you restrict what can be referred to in the template?

For these reasons, I propose a scheme like this:

• the template is given in the marker
• image policies are named in the same format as for simple setters; anything that refers to policies in different namespaces is ignored

This will mean implementing a kyaml filter, since the kyaml setters2 code won't handle templates given in the marker.

[GregoireW]

This issue resurface if we extract additional attributes with this: fluxcd/image-reflector-controller#211

Has been discussed quickly in the weekly meeting ( I'm not really good at remembering name ... sorry @relu ? or @stefanprodan ? )

This PR: #274 will be reworked to remove the duplicate strategy. but there is some hint on how to manage those attributes.

Today, kyaml is used to replace value by resolving a json from comment :

• {"$imagepolicy": "<policy-namespace>:<policy-name>"}
• {"$imagepolicy": "<policy-namespace>:<policy-name>:tag"}
• {"$imagepolicy": "<policy-namespace>:<policy-name>:name"}

If we think about additional attributes the basic thing to do would be to add some possible attributes like:

• {"$imagepolicy": "<policy-namespace>:<policy-name>:<attributes-name>"}

Today, the kyaml and especially the setter2 package is used with the setter mode. It is like we fill a key/value data map then when the controller find a comment containing a json with an attribute named "$imagepolicy" then the value of this json attribute is used as key on the data map and the value from this data map is set in the kubernetes object attribute.

This will not solve some use case like this issue where a pattern is needed. Also, the format of the value is a little bit strange.

The kyaml (setter2) package as also a substitution part. This mode need us to declare some template ( template A = the pattern + a substitution map). The process loop on each key of the substitution map, and substitute this key string in the pattern. But not directly with a value, The substitution map value is not a real value but a reference to previous "setter" data map. In short, the template contains reference of reference.

For me it would be a really strange way to manage this. it would mean declare a template somewhere (how ? in a specific comment ?) knowing the content would be a little bit strange to write (reference to value need to be the reference to the reference, so something like "<policy-namespace>:<policy-name>:<attributes-name>". Not really user friendly ... ( not sure we want user to have template mixing attributes from different policy).

I would like to use the additional attributes to change a little bit how this is managed.

I was thinking of comment in the form of

• {"$imagepolicy": "<policy-namespace>:<policy-name>:tag"} ==> like today, the tag
• {"$imagepolicy": "<policy-namespace>:<policy-name>:name"} ==> like today, the image name without tag
• {"$imagepolicy": "<policy-namespace>:<policy-name>" attribute="xx"} ==> the attribute xx
• {"$imagepolicy": "<policy-namespace>:<policy-name>" template="a-{{.xx}}-value"} ==> a Go template, where the data map will contains tag, name, and additional attributes
• {"$imagepolicy": "<policy-namespace>:<policy-name>"} ==> like today: image name and tag

The case where the json contains template and attribute should fail and end up as error.

Doing this means the current way of using kyaml will be completely modified. It will be a little bit like I did in the #274 Before, the process was relying completely on kyaml to set data. After, the process rely on parsing correctly the json AND set correctly the value ( is kyaml check if the data is correct before setting the value ? like check is replicas is an integer, if a name do not start with '-' ... ? )

Question is : How to proceed ?

[relu]

@GregoireW as @stefanprodan already suggested, please follow the RFC process and I would gladly sponsor it 👍

From my point of view, there are two RFCs that can be extracted from your current proposed changes:

1. Tag attribute extraction (done under IRC)
2. Setters substitutions/templating (done under IAC)