kubelet_volume_stats_* metrics not available in 1.34

[maciej-p]

Bug Report

When using Kubernetes version 1.34 kubelet_volume_stats_* metrics such as kubelet_volume_stats_available_bytes are no longer present

Description

More details here: kubernetes/kubernetes#133847

Issue looks resolved in Kubernetes version 1.34.2

Environment

• Talos version: 1.11.5
• Kubernetes version: 1.34.1

[frezbo]

Why is this a talos issue? Just need to update via upgrade-k8s.

[maciej-p]

Because of this: https://github.com/siderolabs/talos/blob/release-1.11/pkg/machinery/constants/constants.go#L399

and TalosControlPlane

spec:
  version: v1.34.2

not works.

upgrade-k8s will upgrade cluster but Cluster API will keep only what is released in Talos 1.11.5 (v1.34.1)

[frezbo]

not sure I follow, what's relation b/w talos and cluster api, talos can be installed with any supported k8s version, don't see any issue here

[maciej-p]

@frezbo The issue is specific to Cluster API deployments with TalosControlPlane.

When managing Talos clusters via CAPI, the TalosControlPlane spec has a version field that's meant to declare the desired Kubernetes version:

spec:
  version: v1.34.2

However, the control-plane provider doesn't actually swap Kubernetes binaries on the nodes—it only tracks the version as metadata for rollout decisions. The actual Kubernetes version running on nodes comes from whatever is bundled in the Talos image.

Since Talos 1.11.5 ships with Kubernetes v1.34.1, setting spec.version: v1.34.2 in the TalosControlPlane resource has no effect—the nodes stay on v1.34.1, which has the missing kubelet_volume_stats_* metrics bug.

Running talosctl upgrade-k8s manually can upgrade to v1.34.2, but CAPI will drift the cluster back to v1.34.1 on the next reconciliation since that's what the Talos 1.11.5 image contains.

To properly fix this in a CAPI-managed cluster, we need a Talos release (e.g., 1.11.6) that bundles Kubernetes v1.34.2.

This explains the CAPI-specific behavior and why it's a Talos packaging issue rather than something you can work around with upgrade-k8s.

[frezbo]

this is expected behavior, new nodes would only get updates, 1.11.6 would be released in time for security fixes

[frezbo]

so either scale down one node at a time and scale up, or add new nodes if there's available