Skip to content

Commit

Permalink
noxfile.py: On Nix, auto-patch binaries installed by Nox
Browse files Browse the repository at this point in the history
On Nix/NixOS, when Nox installs a dependency into its own virtualenv,
any pre-built binaries that are installed will break hermeticity (and on
NixOS w/o nix-ld, they will fail to execute at all).

Nix usually solves this by running patchElf when installing pre-built
binaries, to point them to the Nix-provided (and thus hermetic) loader +
shared libs.

But what to do when the pre-built binaries are not provided by Nix?
We have already solved this for binaries installed by Poetry virtualenv:
In our shell.nix, we run `autoPatchelf` on the Poetry virtualenv after
Poetry has populated it.

This commit does the same for the virtualenvs setup/managed by Nox: When
install_groups() has finished installing the dependencies into the
virtualenv provided by Nox, we check to see if we're running on Nix
(specifically, we check if auto-patchelf-hook is in our $buildInputs,
which signifies that we're running inside our nix-shell), and then we
use the same nix-shell to run `autoPatchelf` on the Nox virtualenv.

This allows us to use Python dependencies with pre-built binaries, not
only in the Poetry dev environemnt, but also in the test environments
managed by Nox.
  • Loading branch information
jherland committed Mar 18, 2024
1 parent 5460154 commit 4a4a1aa
Showing 1 changed file with 21 additions and 0 deletions.
21 changes: 21 additions & 0 deletions noxfile.py
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import hashlib
import os
from pathlib import Path
from typing import Iterable

Expand All @@ -7,6 +8,23 @@
python_versions = ["3.7", "3.8", "3.9", "3.10", "3.11", "3.12"]


def patch_binaries_if_needed(session: nox.Session, venv_dir: str) -> None:
"""If we are on Nix, auto-patch any binaries under `venv_dir`.
Detect if we are running under Nix, and auto-patch any pre-built binaries
that were just installed into the Nox virtualenv.
"""
build_inputs = os.environ.get("buildInputs", "") # noqa: SIM112
if "auto-patchelf-hook" not in build_inputs:
return

# We want to invoke autoPatchelf, but it is a shell function in the
# surrounding Nix shell, and thus not directly available to session.run().
# However, we can invoke nix-shell and tell it to run autoPathelf for us:
argv = ["nix-shell", "--run", f"autoPatchelf {venv_dir}"]
session.run(*argv, silent=True, external=True)


def install_groups(
session: nox.Session,
*,
Expand Down Expand Up @@ -61,6 +79,9 @@ def install_groups(
if include_self:
session.install("-e", ".")

if not session.virtualenv._reused:
patch_binaries_if_needed(session, session.virtualenv.location)


@nox.session(python=python_versions)
def tests(session):
Expand Down

0 comments on commit 4a4a1aa

Please sign in to comment.