Skip to content

cloud-hypervisor: Problems running multiple microvms because cid is hardcoded #378

@Ramblurr

Description

@Ramblurr

Problem

The microvm cloud-hypervisor runner is hardcoding the cid for vsock at 3.

That isn't good because:

  • cids should be unique per vm
  • it makes the microvm.vsock.cid useless for chv

https://github.com/astro/microvm.nix/blob/f30ae9b9e740116d044935c9c0c68bd1c2ce8432/lib/runners/cloud-hypervisor.nix#L153

I suppose hardcoded vsock was added to support systemd-notify, but it ignores the user's configuration and causes problems when running multiple VMs from running on the same host.

Solution

The fix is to:

  1. Respect microvm.vsock.cid when set
  2. Extract and merge any user-provided --vsock options from extraArgs
  3. Only use the hardcoded default when neither is specified
  4. Throw a clear error if both microvm.vsock.cid and --vsock cid=... are provided

This allows users to either:

  • Set microvm.vsock.cid = 5; in their config
  • Pass --vsock "socket=/custom/path.sock" via extraArgs
  • Get the default behavior if they don't configure vsock

The implementation properly handles partial user options (e.g., just cid or just socket) and merges them with sensible defaults, while maintaining backward compatibility for users who don't configure vsock at all.

I guess this is a breaking change for users who were relying on the automatic systemd-notify support, but I'm not sure that will really affect runtime?

We could add a text in the release like:

Previously, cloud-hypervisor would always configure vsock with cid=3 and enable systemd-notify notifications automatically. With this change, users must explicitly set microvm.vsock.cid to enable the notify feature.

To maintain the previous behavior, users should add to their configuration:

microvm.vsock.cid = 3;  # or any unique CID for your VM

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions