Down builder brings all builds to a crawl

[sellout]

Describe the bug

I set up distributed builds as described in the NixOS wiki and various other places, with multiple builders listed.

If one of the builders isn’t accepting connections for some reason (especially a builder with a high priority), it will hold up the process with a long wait for a timeout. This isn’t so bad in itself, except that nix doesn’t remember that the connection timed out a second ago, so it tries again for each derivation that needs to be built.

Expected behavior

Ideally, nix wouldn’t retry a failing builder in the same nix call (at least not without some kind of delay). Without that, some way to disable a single builder on the command line (--builders "" is too scorched earth, and rebuilding the builders sans the problematic one is annoying), so something like --skip-builders "ssh://my-flaky-machine" would be helpful. And barring that … how can I make the connection timeout shorter?

Metadata

nix-env (Nix) 2.24.14

Additional context

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.