lib/test-driver: fix reading command output for other distros

[am-on]

When running the test-driver on other distros (e.g., via https://github.com/numtide/nix-vm-test/) executing a command can fail if the driver receives both stdout and stderr.

The test-driver’s logic for reading output currently assumes it will only read base64 encoded stdout, so any extra stderr content leads to unexpected failures.

This commit fixes the issue by redirecting the stderr to /dev/null, ensuring the driver only sees stdout.

The tradeoff is losing stderr messages in the machine logs. Users can work around this by redirecting stderr to stdout in the command they are sending to the machine:

machine.execute("some_command 2>&1")

Related issues:

• Test driver crashes if special characters are present numtide/nix-vm-test#84
• apt-get install breaks the test runner numtide/nix-vm-test#5

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 25.05 Release Notes (or backporting 24.11 and 25.05 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[am-on]

Worked on this during Thaiger sprint: Thaigersprint/thaigersprint-2025#1

[tfc]

@ofborg test nat.firewall networking.scripted.link installer.simpleProvided installer.separateBootFat networking.scripted.virtual printing keymap.azerty keymap.dvorak-programmer boot-stage1 installer.swraid keymap.neo nfs4.simple i3wm udisks2 networking.networkd.loopback containers-ip ecryptfs login installer.simpleLabels php.httpd zfs.installer predictable-interface-names.unpredictable predictable-interface-names.unpredictableNetworkd mutableUsers

[tfc]

@ofborg test pantheon boot.biosCdrom boot.uefiUsb networking.networkd.static latestKernel.login networking.networkd.dhcpSimple networking.networkd.sit env switchTest hibernate predictable-interface-names.predictable gnome networking.networkd.bridge networking.networkd.privacy plasma5 systemd-networkd-ipv6-prefix-delegation lightdm installer.simple installer.btrfsSimple installer.btrfsSubvols sway networking.networkd.link networking.scripted.privacy

[tfc]

@ofborg test nfs3.simple php.fpm fontconfig-default-fonts networking.scripted.bond nano keymap.dvorak installer.luksroot networking.scripted.bridge docker simple chromium networking.scripted.dhcpOneIf networking.networkd.bond php.pcre nat.standalone openssh networking.scripted.vlan ipv6 installer.separateBoot misc predictable-interface-names.predictableNetworkd networking.scripted.loopback keymap.colemak installer.btrfsSubvolDefault

[jfly]

I still think the previous behavior is better. IMO, the real fix is to make sure we're always invoking qemu/using the serial console in the same way (namely, where stderr gets printed to qemu's output, rather than the socket the test driver reads).

[jfly]

Dug into this a bit with @Mic92's help:

• Here's where the "backdoor root shell" redirects its stderr: https://github.com/nixos/nixpkgs/blob/6d7f6a92cc5830f28dc56e4de8a4dc18c7ceda69/nixos/modules/testing/test-instrumentation.nix#L45
• There's analogous code in nix-vim-test here: https://github.com/numtide/nix-vm-test/blob/cfaa0ac4b085866d18afc886971b9e89ea662e90/generic/default.nix#L57

I tried spinning up a nix-vm-test myself (following these instructions) and was unable to reproduce the issue you're hitting, so I'm a little confused. Would be nice to have a simple repro to poke at.