Skip to content

Conversation

@dcarley
Copy link
Contributor

@dcarley dcarley commented Jun 6, 2025

This took a while to figure out because "building" a Python application
normally means distributing for it installation with a tool like pip
but it doesn't necessarily produce a standalone artifact that can be run
as a "binary" command, in the same way that bundler can produce
binstubs for Ruby applications.

Poetry is instructed to install into a new virtualenv within $out:

% l result-quotes-app-python/quotes-app-python-Xe7YT7dH-py3.12
total 12K
-r--r--r--  1 root wheel  40 Jan  1  1970 .gitignore
-r--r--r--  1 root wheel 194 Jan  1  1970 CACHEDIR.TAG
dr-xr-xr-x 14 root wheel 448 Jan  1  1970 bin
dr-xr-xr-x  3 root wheel  96 Jan  1  1970 lib
-r--r--r--  1 root wheel 465 Jan  1  1970 pyvenv.cfg

I can't control the nesting though so we have to symlink the binary:

% l result-quotes-app-python/bin/quotes-app-python
lrwxr-xr-x 1 root wheel 58 Jan  1  1970 result-quotes-app-python/bin/quotes-app-python -> ../quotes-app-python-Xe7YT7dH-py3.12/bin/quotes-app-python

That binstub references the virtualenv:

% cat result-quotes-app-python/bin/quotes-app-python
#!/nix/store/bw0gjnj5d2fplfrci7m1sp2z9si8xkll-quotes-app-python-0.0.1/quotes-app-python-Xe7YT7dH-py3.12/bin/python
import sys
from quotes_app_python.main import main

if __name__ == '__main__':
    sys.exit(main())

The resulting build can be run without being in an existing activation:

% ./result-quotes-app-python/bin/quotes-app-python
INFO:quotes-app-python:Loaded 10 quotes
INFO:quotes-app-python:Starting server on http://0.0.0.0:3000/
INFO:     Started server process [24865]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://0.0.0.0:3000/ (Press CTRL+C to quit)

The resulting closure looks like this:

% nix run nixpkgs#nix-tree -- --dot ./result-quotes-app-python
strict digraph {
  "sqlite-3.48.0" -> "zlib-1.3.1" [];
  "quotes-app-python-0.0.1" -> "python3-3.12.10" [];
  "readline-8.2p13" -> "ncurses-6.5" [];
  "python3-3.12.10" -> "ncurses-6.5" [];
  "python3-3.12.10" -> "mpdecimal-4.0.0" [];
  "python3-3.12.10" -> "sqlite-3.48.0" [];
  "python3-3.12.10" -> "xz-5.8.1" [];
  "python3-3.12.10" -> "bash-5.2p37" [];
  "python3-3.12.10" -> "libffi-39" [];
  "python3-3.12.10" -> "gdbm-1.25-lib" [];
  "python3-3.12.10" -> "tzdata-2025b" [];
  "python3-3.12.10" -> "readline-8.2p13" [];
  "python3-3.12.10" -> "bzip2-1.0.8" [];
  "python3-3.12.10" -> "expat-2.7.1" [];
  "python3-3.12.10" -> "zlib-1.3.1" [];
  "python3-3.12.10" -> "mailcap-2.1.54" [];
  "python3-3.12.10" -> "libxcrypt-4.4.38" [];
  "python3-3.12.10" -> "openssl-3.4.1" [];
}

The build produces a handful of warnings if run without previously
running flox activate due to this newly filed bug:

Also filed a bug for detecting newer Poetry projects:

dcarley added 6 commits June 6, 2025 14:52
The application code is the result of o4-mini-high converting the Rust
implementation, with some additional modifications to tidy it up.

The `pyproject.toml` is the result of running `poetry init`.
So that a binstub is rendered for `quotes-app-python` in the venv.
So that the Flox auto-setup hook indetifies the project as Poetry,
otherwise we only get Python installed.
This uses the auto-setup hook, which has identified incompat with build
in the past. The build stanza will be added in a separate commit.
Re-generated with `nix run github:flox/flox -- init` to pick up changes
that are currently only on `main`.
This took a while to figure out because "building" a Python application
normally means distributing for it installation with a tool like `pip`
but it doesn't necessarily produce a standalone artifact that can be run
as a "binary" command, in the same way that `bundler` can produce
binstubs for Ruby applications.

Poetry is instructed to install into a new virtualenv within `$out`:

    % l result-quotes-app-python/quotes-app-python-Xe7YT7dH-py3.12
    total 12K
    -r--r--r--  1 root wheel  40 Jan  1  1970 .gitignore
    -r--r--r--  1 root wheel 194 Jan  1  1970 CACHEDIR.TAG
    dr-xr-xr-x 14 root wheel 448 Jan  1  1970 bin
    dr-xr-xr-x  3 root wheel  96 Jan  1  1970 lib
    -r--r--r--  1 root wheel 465 Jan  1  1970 pyvenv.cfg

I can't control the nesting though so we have to symlink the binary:

    % l result-quotes-app-python/bin/quotes-app-python
    lrwxr-xr-x 1 root wheel 58 Jan  1  1970 result-quotes-app-python/bin/quotes-app-python -> ../quotes-app-python-Xe7YT7dH-py3.12/bin/quotes-app-python

That binstub references the virtualenv:

    % cat result-quotes-app-python/bin/quotes-app-python
    #!/nix/store/bw0gjnj5d2fplfrci7m1sp2z9si8xkll-quotes-app-python-0.0.1/quotes-app-python-Xe7YT7dH-py3.12/bin/python
    import sys
    from quotes_app_python.main import main

    if __name__ == '__main__':
        sys.exit(main())

The resulting build can be run without being in an existing activation:

    % ./result-quotes-app-python/bin/quotes-app-python
    INFO:quotes-app-python:Loaded 10 quotes
    INFO:quotes-app-python:Starting server on http://0.0.0.0:3000
    INFO:     Started server process [24865]
    INFO:     Waiting for application startup.
    INFO:     Application startup complete.
    INFO:     Uvicorn running on http://0.0.0.0:3000 (Press CTRL+C to quit)

The resulting closure looks like this:

    % nix run nixpkgs#nix-tree -- --dot ./result-quotes-app-python
    strict digraph {
      "sqlite-3.48.0" -> "zlib-1.3.1" [];
      "quotes-app-python-0.0.1" -> "python3-3.12.10" [];
      "readline-8.2p13" -> "ncurses-6.5" [];
      "python3-3.12.10" -> "ncurses-6.5" [];
      "python3-3.12.10" -> "mpdecimal-4.0.0" [];
      "python3-3.12.10" -> "sqlite-3.48.0" [];
      "python3-3.12.10" -> "xz-5.8.1" [];
      "python3-3.12.10" -> "bash-5.2p37" [];
      "python3-3.12.10" -> "libffi-39" [];
      "python3-3.12.10" -> "gdbm-1.25-lib" [];
      "python3-3.12.10" -> "tzdata-2025b" [];
      "python3-3.12.10" -> "readline-8.2p13" [];
      "python3-3.12.10" -> "bzip2-1.0.8" [];
      "python3-3.12.10" -> "expat-2.7.1" [];
      "python3-3.12.10" -> "zlib-1.3.1" [];
      "python3-3.12.10" -> "mailcap-2.1.54" [];
      "python3-3.12.10" -> "libxcrypt-4.4.38" [];
      "python3-3.12.10" -> "openssl-3.4.1" [];
    }

The build produces a handful of warnings if run without previously
running `flox activate` due to this newly filed bug:

- flox/flox#3203
@dcarley dcarley requested a review from a team June 6, 2025 14:26
Copy link
Member

@zmitchell zmitchell left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Heroic effort to figure this bullshit out

@dcarley dcarley merged commit 77a0506 into main Jun 9, 2025
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants