Skip to content

feat(import): implement pixi import for environment.yml #4096

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 13 commits into
base: main
Choose a base branch
from

Conversation

lucascolley
Copy link
Contributor

@lucascolley lucascolley commented Jul 7, 2025

closes gh-4085

TODO:

Copy link
Contributor

@Hofer-Julian Hofer-Julian left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like a good start!

@lucascolley lucascolley force-pushed the import branch 2 times, most recently from c7b296b to b13d8a6 Compare July 7, 2025 15:22
@lucascolley
Copy link
Contributor Author

lucascolley commented Jul 8, 2025

It sounds like we are initially interested in attempting to process these file types as these formats:

Input File Types Format
*.yml conda environment.yml
*.txt conda-lock.yml
... conda requirements.txt
PyPI requirements.txt
...

Option 1: just have --conda and --pypi flags (2 flags needed to distinguish formats for txt files (I think? Can one txt file be valid as input to both conda and pip?))

  • pixi import --conda:
    • attempt to process yml files as environment.yml format, fall-back to conda-lock.yml format if it fails
    • attempt to process txt files like conda create --file
    • error for other file types (but scope to grow)
  • pixi import --pypi:
    • attempt to process txt files like pip
    • error for other file types (but scope to grow)
  • pixi import:
    1. we could make this error and force a flag to be provided
    2. we could instead process yml files like pixi import --conda, and force a flag to be provided for txt files (this seems better to me)
    3. we could instead default to conda for all files (I think this may cause user confusion for pixi import requirements.txt)
  • scope to grow more flags in the future

Is it a problem if one cannot force only trying conda-lock.yml format?


Option 2: just have a --format flag:

  • pixi import --format conda-env:
    • attempt to process file as a conda environment.yml
  • pixi import --format conda-lock:
    • attempt to process file as a conda-lock.yml
  • pixi import --format conda-txt:
    • attempt to process txt file like conda create --file
  • pixi import --format pypi-txt:
    • attempt to process txt file like pip
  • pixi import:
    1. we could make this error and force a format to be provided
    2. we could instead try to process yml files like --format conda-env, then try --format conda-lock, and force a format to be provided for txt files (this seems better to me)
    3. we could also default to conda for txt files (I think this may cause user confusion)
  • scope to grow more formats in the future

Alternative suggestions:

  • some combination of these 2 options: one could allow pixi import --conda as well as pixi import --format=conda-lock (this feels like it introduces a bit of redundancy to me, even if no two commands provide identical behaviour for all file formats)
  • ...

What do you think @Hofer-Julian @baszalmstra ?

@Hofer-Julian
Copy link
Contributor

Nice write-up @lucascolley!

Seeing it written out like this, option 2 seems like the way to go.
It's obvious and easy to extend without breaking existing workflow.

  • pixi import:
    1. we could make this error and force a format to be provided
    2. we could instead try to process yml files like --format conda-env, then try --format conda-lock, and force a format to be provided for txt files (this seems better to me)
    3. we could also default to conda for txt files (I think this may cause user confusion)
  • scope to grow more formats in the future

What I like about 2. is that we can also sync the behavior with pixi init --import /path/to/file.

@lucascolley lucascolley marked this pull request as ready for review July 9, 2025 13:27
@lucascolley
Copy link
Contributor Author

@Hofer-Julian could you check whether these tests are along the right lines?

If so, probably just a few more edge cases to cover, as well as implementing handling for file.variables().

@Hofer-Julian
Copy link
Contributor

@Hofer-Julian could you check whether these tests are along the right lines?

If so, probably just a few more edge cases to cover, as well as implementing handling for file.variables().

The tests look good, great work!
The biggest missing thing I see is that you should also compare that the dependencies in the pixi.toml actually match the input file.

@lucascolley
Copy link
Contributor Author

The biggest missing thing I see is that you should also compare that the dependencies in the pixi.toml actually match the input file.

Added a test for that just now — I think this is all ready from my side! We can save converting variables to activation.env for a follow-up (#4103).

Copy link
Contributor Author

@lucascolley lucascolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I should add some docs as well to explain how the parameters interact

Copy link
Contributor Author

@lucascolley lucascolley left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nichmor perhaps you could take a look at this PR while Julian is at EuroPython?

Comment on lines +116 to +142
for spec in conda_deps {
// Determine the name of the package to add
let (Some(package_name), spec) = spec.clone().into_nameless() else {
miette::bail!(
"{} does not support wildcard dependencies",
pixi_utils::executable_name()
);
};
let spec = PixiSpec::from_nameless_matchspec(spec, &channel_config);
workspace.manifest().add_dependency(
&package_name,
&spec,
SpecType::Run,
&platforms,
&feature_name,
DependencyOverwriteBehavior::Overwrite,
)?;
}
for requirement in pypi_deps {
workspace.manifest().add_pep508_dependency(
(&requirement, None),
&platforms,
&feature_name,
None,
DependencyOverwriteBehavior::Overwrite,
None,
)?;
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this is a modified version of

pixi/src/cli/init.rs

Lines 322 to 351 in 6359d0a

for spec in conda_deps {
// Determine the name of the package to add
let (Some(name), spec) = spec.clone().into_nameless() else {
miette::bail!(
"{} does not support wildcard dependencies",
pixi_utils::executable_name()
);
};
let spec = PixiSpec::from_nameless_matchspec(spec, &channel_config);
workspace.manifest().add_dependency(
&name,
&spec,
SpecType::Run,
// No platforms required as you can't define them in the yaml
&[],
&FeatureName::default(),
DependencyOverwriteBehavior::Overwrite,
)?;
}
for requirement in pypi_deps {
workspace.manifest().add_pep508_dependency(
(&requirement, None),
// No platforms required as you can't define them in the yaml
&[],
&FeatureName::default(),
None,
DependencyOverwriteBehavior::Overwrite,
None,
)?;
}
.

Could maybe be refactored out to a new function.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add pixi import environment.yml
3 participants