[feature] Setting Omni machine labels at boot time (cloud-init)

[ecv]

Problem Description

I'm an infrastructure engineer at an alt-cloud service provider, and I have several Omni clusters with a few compute providers in various locations. Machines boot in these locations with labels that associate them with machine-classes. (Specifically, 'provider' and 'site', at the moment.)

These providers all accept the nocloud (cloud-init) Talos ISO, and I pass my machine-config in as an 'initialization script', which is actually cloud-config underneath the hood.

I have not been able to find a way to pass Omni labels in at boot time using this initialization-script or the machine-config. The machine-config time does not seem to support my setting labels.

So, I am posting to the providers one ISO for each provider-location. This scales linearly with my available compute zones. It's a bit of a maintenance burden, and is especially taxing on providers, whose ISO ingestion processes are in various states of automation.

Solution

I'd like to be able to set arbitrary labels in the machine-config.

Then, with two lines of YAML, I could throw my entire ISO management apparatus in the river, and the support desks at my compute providers will breathe a little easier.

Alternative Solutions

I don't believe I can run any executables at boot time to, say, hit a private API to learn facts about their place in the world. So, this limits my alternatives. However I also believe that's the right approach, we should rely on nocloud/cloud-init interfaces for these things.

You can, however, probably pull more tags out of no-cloud metadata. If you did this, we might be fortunate and find the providers encoding their own site data in their cloud-config.

Notes

There's a thread in your Slack chat with Datum, this link works in my client, unsure if it's portable to other users and views.

If these labels also became available in Kubernetes that would be cool, but I need Omni labeling specifically to solve my cluster join.

[steverfrancis]

There is some discussion of this at siderolabs/talos#9895

You should be able to:
You can use kubelet's registerWithTaints configuration (and pass it via machine.kubelet.extraConfig to taint at the moment the Node is created. Updating taints on the fly is still denied by default.

Does that address issue?

[ecv]

I'll experiment and report, it might.

[ecv]

I think it won't. These are Kubernetes labels, and I need Omni labels to affect machine-class join.

Adjusted my issue to better clarify that I am after Omni labeling.

[Unix4ever]

I think currently it's only doable if you generate separate images per machine.
Encoding Omni machine labels into the ISO is possible.