Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[conda] Support Pixi's use of pyproject.toml to declare conda dependencies? #453

Closed
jherland opened this issue Sep 6, 2024 · 1 comment · Fixed by #455
Closed

[conda] Support Pixi's use of pyproject.toml to declare conda dependencies? #453

jherland opened this issue Sep 6, 2024 · 1 comment · Fixed by #455
Labels
integration Integrating FawltyDeps with other tools needs-real-projects-test This issue is more easily tackled once we have a project in `real_project` that illustrate the issue research-needed type: feature request

Comments

@jherland
Copy link
Member

jherland commented Sep 6, 2024

Thanks to @ctcjab's comment for highlighting Pixi's use of pyproject.toml in conda projects. Looking at https://pixi.sh/latest/advanced/pyproject_toml/ Pixi allows a combinations of PyPI dependencies and Conda dependencies to be declared in pyproject.toml:

AFAIUI, PyPI dependencies are declared in the project.dependencies section of pyproject.toml, and seems to obey PEP621. (This part should thus already be supported by FawltyDeps.)

Next, Conda dependencies are declared in a [tool.pixi.dependencies] section. In the case of overlaps (a dependency name that occurs in both sections), the Conda dependency is preferred and the PyPI dependency is ignored.

The dependencies specifiers in the [tool.pixi.dependencies] section initially appear to be formatted in the same as PEP621's project.dependencies (see https://packaging.python.org/en/latest/specifications/dependency-specifiers/ for details), but according to https://pixi.sh/latest/reference/project_configuration/#the-dependencies-tables there are slight differences in these formats:

[Conda] Dependencies are defined using a VersionSpec.
[...]
[PyPI] dependencies don't follow the conda matchspec specification. The version is a string specification of the version according to PEP404/PyPA. Additionally, a list of extra's can be included, which are essentially optional dependencies. Note that this version is distinct from the conda MatchSpec type.

It is not yet clear to me whether the conda version specifiers are a subset of the PyPI version specifiers (and can thus reuse the same parser), or if a separate parser is necessary.

What does seem clear is that the "pinned" version specifiers that specify an exact package build elsewhere in the Conda ecosystem (e.g. the requests=2.32.3=py38h06a4308_0 string that would appear in a conda-generated environment.yml or requirements.txt) are NOT allowed by Pixi here.


Note that Pixi (like Conda) also supports non-Python projects (which is outside FawltyDeps' scope), and also have their own pixi.toml file used (instead of pyproject.toml) to configure Pixi projects. Supporting pixi.toml in FawltyDeps might be a valuable feature at some point, but is outside the scope of this issue.

@jherland jherland added type: feature request research-needed needs-real-projects-test This issue is more easily tackled once we have a project in `real_project` that illustrate the issue integration Integrating FawltyDeps with other tools labels Sep 6, 2024
@jherland jherland linked a pull request Sep 10, 2024 that will close this issue
@jherland
Copy link
Member Author

PR #455 adds support for parsing the various Pixi fields in pyproject.toml.

Please note that this does not introduce any differentiation between Conda dependencies and PyPI dependencies, meaning that we for now proceed on the flawed assumption that a Conda dependency named FOO will map one-to-one to a Python package named FOO. This is certainly not true for Conda dependencies that are not Python packages, and it probably isn't even true for all Conda dependencies that do include Python packages.

The PR also bypasses the issue of how to parse Conda dependency version specifiers, merely by not parsing them at all. FalwtyDeps has so far not cared about versions of dependencies anyway, and hopefully we can get away with the same for Conda dependencies

Still, this is at least one way to dip our toe into the Conda waters. Next, we can add support for parsing pixi.toml and then environment.yml. And we can then revisit the flawed conda==python assumption in later PRs.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
integration Integrating FawltyDeps with other tools needs-real-projects-test This issue is more easily tackled once we have a project in `real_project` that illustrate the issue research-needed type: feature request
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant