Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Dead batteries packages: standard-aifc, standard-sunau, standard-chunk, and audioop-lts #29400

Merged
merged 3 commits into from
Mar 12, 2025

Conversation

bmcfee
Copy link
Contributor

@bmcfee bmcfee commented Mar 11, 2025

Checklist

  • Title of this PR is meaningful: e.g. "Adding my_nifty_package", not "updated meta.yaml".
  • License file is packaged (see here for an example).
  • Source is from official source.
  • Package does not vendor other packages. (If a package uses the source of another package, they should be separate packages or the licenses of all packages need to be packaged).
  • If static libraries are linked in, the license of the static library is packaged.
  • Package does not ship static libraries. If static libraries are needed, follow CFEP-18.
  • Build number is 0.
  • A tarball (url) rather than a repo (e.g. git_url) is used in your recipe (see here for more details).
  • GitHub users listed in the maintainer section have posted a comment confirming they are willing to be listed there.
  • When in trouble, please check our knowledge base documentation before pinging a team.

Copy link
Contributor

Hi! This is the staged-recipes linter and your PR looks excellent! 🚀

@conda-forge-admin
Copy link
Contributor

conda-forge-admin commented Mar 11, 2025

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipes/standard-chunk/meta.yaml, recipes/audioop-lts/meta.yaml, recipes/standard-sunau/meta.yaml, recipes/standard-aifc/meta.yaml) and found it was in an excellent condition.

@bmcfee
Copy link
Contributor Author

bmcfee commented Mar 11, 2025

@conda-forge/help-python 👋 I could use some input on how this particular set of packages should work in conda-forge.

Summary of the problem:

  • This PR includes four "dead batteries" packages that were removed from the standard library in python 3.13, all relating to audio processing. (aifc, sunau, audioop, and chunk). These packages are now hosted externally on pypi to provide continued support for downstream libraries, some of which exist in the conda-forge space already (audioread and librosa, notably).
  • I've stubbed out the package meta.yaml's according to grayskull and smithy linting, but I suspect the default recommendations are not going to be correct here.

The major problem that I'm anticipating is that these packages collide with standard namespace packages in python <=3.12. (That is, standard-aifc provides the aifc package, and so on.) On PyPI, this is skirted around by adding a python>='3.13' constraint on the requirements so that these packages are not installed on older pythons.

To my understanding, we can do this in conda-forge as well, but not with noarch packages. This effectively means that any downstream package that depends on these would have to stop using noarch because we can't use selectors in the requirements spec.

Any input on how to resolve this situation would be much appreciated.

sha256: 64e249c7cb4b3daf2fdba4e95721f811bde8bdfc43ad9f936589b7bb2fae2e43

build:
script: {{ PYTHON }} -m pip install . -vv --no-deps --no-build-isolation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one can probably be noarch, right?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In principle yes, but see my comment/question in the main PR thread about noarch vs selectors #29400 (comment)

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, missed that, you need to set the min python variable, add

{% set python_min = "3.13" %}

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This effectively means that any downstream package that depends on these would have to stop using noarch because we can't use selectors in the requirements spec.

Ah, I misread. OK. If this module is done "right" you probably can install it without clobbering, it will become a stub. If not, you will have to keep this as-is.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this module is done "right" you probably can install it without clobbering

To clarify, do you mean the implementation of the module, or the packaging of the module?

My understanding is that the module code itself is a direct extraction from old standard lib code, and doesn't have the smarts to stub itself out. At the same time, I don't think the deadlib maintainers are open to contributions that would add the stub logic, since they want to keep the code essentially as archival.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Quick follow-up:

  • I checked the deadlib aifc source, and verify that it's not doing anything smart to detect collisions (ie that it's running on an old python)
  • OTOH, I also installed it via pip in a python 3.7 environment, and it seems to work. That is, import aifc hits the standard lib module first and does not try to load the deadlib version. I expect that even if it did hit the deadlib version, it would still work, but I'd rather not chance it.

This then raises the question: is it legit to package these deadlibs as noarch that can be installed on any python version? Maybe that's a question for the deadlib maintainers?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This then raises the question: is it legit to package these deadlibs as noarch that can be installed on any python version? Maybe that's a question for the deadlib maintainers?

Giving the fact that is clobbers existing files? No. It is better too sacrifice noarch advantages here. Let's keep these as is. Thanks for the clarifications.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Giving the fact that is clobbers existing files?

It's not clobbering the existing files, but "correct" behavior (ie using the standard lib implementation instead of the external package) depends on the module import search order. While this is (to my knowledge) well-defined, it does make me a little nervous.

It is better too sacrifice noarch advantages here.

For these packages, that's fine. I'm more concerned about the downstream packages that may want to stay noarch and have these deadlibs as dependencies if needed. This is where I'm stuck, and I don't see a simple path forward with the usual packaging pipeline.

I do wonder if it's possible to do something clever here: what if these deadlibs are not built as noarch, and we create stub packages for python <=3.12 that do not install anything? For 3.13+ the packages would install as we currently have them in this PR.

This would make it so that the downstream dependents could still be noarch and depend on these libs, but installing, say, standard-aifc in a 3.12 environment would act as a no-op.

Is such a thing possible?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

While this is (to my knowledge) well-defined, it does make me a little nervous.

Same.

This is where I'm stuck, and I don't see a simple path forward with the usual packaging pipeline.

Yeah, better this one "arched" and many other noarch.

Is such a thing possible?

Probably. Not sure if it is worth it though. IMO, we should fix the "semi-arch" concept so we can do things like this as noarch moving on. However, that is a fix on recipe format and conda itself.

sha256: b319a1ac95a09a2378a8442f403c66f4fd4b36616d6df6ae82b8e536ee790908

build:
script: {{ PYTHON }} -m pip install . -vv --no-deps --no-build-isolation
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe this one can be noarch too.

@ocefpaf ocefpaf merged commit 06bbdc1 into conda-forge:main Mar 12, 2025
7 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

3 participants