AMD GPU support eg. ROCm gpu-operator

[Syntax3rror404]

Feature Request

In the documentation I only find docs about NVIDIA for example (https://www.talos.dev/v1.9/talos-guides/configuration/nvidia-gpu-proprietary/) but nothing about AMD GPUs.

I want to install a bare-metal cluster with AMD GPUs and want to use the AMD GPU operator. https://github.com/[ROCm/gpu-operator](https://github.com/ROCm/gpu-operator). Is this supported?

Description

Question about AMD GPU support.

[smira]

Moved to a discussion, as it's not an issue.

As far as we know, Talos users can use AMD GPUs out of the box, the only requirement should be to install siderolabs/amdgpu extensions.

If it works for you, please contribute the documentation!

[Syntax3rror404]

No because we also need like nvidia additional kernel modules. For example amdgpu-dkms

[frezbo]

amdgpu modules are already included in talos, as said earlier there are RocM users on Talos and it usually just need the amdgpu and ucode extensions

[blake-hamm]

Can confirm it works for me on a bare metal AMD AI max 395+ (strix halo) with this config:

customization:
  extraKernelArgs:
    - amd_iommu=off
    - amdgpu.gttsize=131072
    - ttm.pages_limit=33554432
  systemExtensions:
    officialExtensions:
      - siderolabs/fuse3
      - siderolabs/amd-ucode
      - siderolabs/amdgpu

Then, I deployed the ROCm operator with this config. Everything works great!

However, I am having issues with talos recognizing an AMD AI Pro R9700. This is a bit more complicated because I'm doing PCIe passthrough on a proxmox vm, so I'll continue troubleshooting.

One note on the Strix Halo (and most likely the R9700). This hardware is very new and many people recommend running a kernel newer than 6.16. I know this goes against the talos philosophy, which is on a 6.12 kernel. I noticed these docs for building a custom talos image. Is this the best way to build talos with a newer kernel?

I might try to configure the ROCm operator to manage driver installation. I'm not sure how that would work in talos, but I suspect I could get newer drivers than what's in the extension. Unfortunately, AMD is just now investing in driver development and their software support is lagging behind their hardware releases..