feat: install an environment of a lockfile to a target dir

[gzm55]

This feature makes pixi to install an environment without a workspace and global/exec specs. This can help distribute and setup many complex and reproducible environments fast based on the pixi/conda/uv caches.

The new command pixi deploy reads the all the required information for installing a single environment from a specified .lock file and global/cli configs, and do not try to search a workspace manifest. Another required option is the target prefix folder to install to. The install steps are similar to pixi install command, except that the .lock file will not be updated to the manifest. The conda and pypi packages are both updated to the prefix folder, except some limit on the source packages.

Another new subcommand pixi workspace export split-lockfile splits the pixi.lock of the current workspace into a specified base folder. The packages of each environment and platform are written into <target>/{platform}/{environment}.lock.

[Hofer-Julian]

Hey @gzm55, thanks for your contribution!

This is a pretty big feature.
Did you discuss this with the team beforehand?

[gzm55]

Hey @gzm55, thanks for your contribution!

This is a pretty big feature. Did you discuss this with the team beforehand?

@Hofer-Julian not yet, but i left a similar discussion last week. it seems not too hard to implement, all the needed low-level functions are ready.

[Hofer-Julian]

Hey @gzm55, thanks for your contribution!
This is a pretty big feature. Did you discuss this with the team beforehand?

@Hofer-Julian not yet, but i left a similar discussion last week. it seems not too hard to implement, all the needed low-level functions are ready.

Responded to your discussion

[gzm55]

@ruben-arts @baszalmstra is there some time to talk about this feature?

[ruben-arts]

Sorry for the late response, we're currently working on lots of different features and other work.

We currently don't guarantee fully standalone lockfiles, for example authentication and stuff like that is not embedded.

What is the vision for this feature, did you think about other options for the implementation? What is the precise use-case you want to solve, as in what do you as a user need?

As is I would not feel comfortable to add such a new big feature.

[gzm55]

What is the vision for this feature, did you think about other options for the implementation? What is the precise use-case you want to solve, as in what do you as a user need?

@ruben-arts Thanks for your response, also for your wonderful tools.

We need quickly clone a runtime prefix in different machines, or share the prefix with different developers. The environments are sometimes composed of conda&pypi packages with complicated special version specs. To quickly distribute the environments reproducibly and precisely, we have some options now:

Text spec formats

1. conda explicit spec: only support conda pkg, no pypi
2. conda environment.yaml: support conda&pypi, may resolve to different version on different platforms or with different environment variables

Binary archive formats:

1. conda-pack: tgz with optional unpack.py, can work directly after decompressing
2. pixi-pack: tgz or executable, requires an unpacker(or embedded into the executable)

The text formats (environment.yaml) are can be exchanged quickly, and often used in developing process, but sometimes may introduce minor difference.
The binary formats are reproducible, and often used to deploy the runtime env on clouds or build docker images. where as the size is not comfortable for developing or compare the difference of two prefixes. Also the two formats both cannot take the advantage of the cache&link mechanism of conda and uv.

Then we looked at the rattler lock file format, which is a candidate of the standardize conda lock format. It supports both packages, reproducible and take the use of conda/uv architeche. And all the unpack/authentication functions already works in pixi.

We currently don't guarantee fully standalone lockfiles, for example authentication and stuff like that is not embedded.

We can use pixi auth login to setup the authentication once or use other secret manager to deploy credentials.json.

As is I would not feel comfortable to add such a new big feature.

This pr can be treated as a PoC, we can still continue discuss in another formal issue and split it into small features.