csi-driver-lvm is not topology or capacity aware across nodes (heterogeneous storage clarification)

[Navin009]

Description

Environment

• Kubernetes cluster running on Talos Linux
• CSI driver: metal-stack/csi-driver-lvm
• Node storage: heterogeneous disks (different sizes / performance per node)

Context
The csi-driver-lvm works well on Talos Linux because it bundles LVM tooling inside the CSI containers and does not depend on host-installed LVM binaries. This makes it suitable for minimal and immutable operating systems.

However, during evaluation for clusters with heterogeneous node storage, some limitations became apparent.

Observed Behavior

• Volumes are provisioned as node-local LVM logical volumes
• PVC binding happens after pod scheduling
• The Kubernetes scheduler has no visibility into per-node storage capacity
• If a pod is scheduled to a node without sufficient free LVM space, PVC provisioning fails
• There is no mechanism to guide scheduling based on available storage across nodes

[Gerrit91]

Hey @Navin009. IS this really a bug or is it something we that can be enhanced? I think this driver has always been used for node-local volumes only. What needs to be fixed specifically to resolve this issue for you?

[Navin009]

As i have found there is no mechanism in place to influence Pod scheduling based on the availability of storage capacity across nodes.

For example, consider a cluster with two nodes:
Node 1: Total capacity 25 GB, with 10 GB free
Node 2: Total capacity 60 GB, with 30 GB free

When a PVC requesting 16 GB is created, the logs indicate that the preferred node selected is Node 1. As a result, provisioning fails due to insufficient available space.

Ideally, the PVC (and corresponding Pod) should be scheduled on Node 2, since it has adequate free storage to satisfy the request.

[Gerrit91]

This functionality should be contained in the latest pre-release, which includes the implementation of capacity tracking API.

Here are references:

• https://github.com/kubernetes-csi/external-provisioner?tab=readme-ov-file#capacity-support
• Migrate architecture to run controller in DaemonSet  #128