First try at accommodating "modern" editable python packages #52
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
|
@cjdoris A polite ping re: this PR. |
|
Thanks for this. I've just been experimenting locally and for me, I don't get the Reading the docs for |
|
Actually I'm extra confused now - the dirs in the |
Interesting! Here's what I see, sorry for the extremely long post: TLDR: I get these |
|
I see the same thing you do when installing Something must be different about the way the python package I'm working with is set up, compared to |
|
Ah, OK, so if I make it use hatchling everything works again: |
|
Thanks for the details, that's really helpful. What is the content of the two I think the difference in behaviour between those packages is that they use different build backends. Yours uses setuptools whereas mine uses hatchling which is more modern and easy to use (IMO). So an easy fix for you would be to upgrade that package to build with hatchling (or some other modern backend of your choice). |
The but the |
Yes, makes sense. The solution I came up with for this PR is perhaps a bit inelegant. If you'd prefer not to accept it, I guess some documentation re: the limitations of setuptools and editable packages would be helpful (which I'd be happy to contribute). |
|
Yeah a PR to add a little "known limitations" section to the readme would be appreciated. |
|
I'm not against adding this functionality but doing it robustly seems hard. Here's a possibly nicer way:
This avoids looking at the filesystem again, since the finders are already loaded when python starts. It does rely on internals ( Edit: Equally if you're happy with hatchling I'm happy to park this PR. |
|
Ah just noticed you already tried hatchling, I didn't see that message! |
That does seem nicer! I took a stab at implementing it and it seems to work well. I updated this PR.
The hatchling switch works fine for my case, but I'm hoping to get others using Julia+Python in my organization, so it would be great if |
|
OK I've added some tests and things and merged this, thanks! |
|
@cjdoris Wonderful, thank you! |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The
pyjuliapkgdocs explain that the package will search forjuliapkg.jsonfiles in the following locations:{project}/pyjuliapkgwhere project is as above (depending on your environment).sys.path.Unfortunately, when a Python package is installed in
--editablemode, it appears that its source location is not, sometimes, added tosys.path. (The source location does appear to be added tosys.pathif the--config-settings editable_mode=compatoption is used withpip, but that option is deprecated: https://setuptools.pypa.io/en/latest/userguide/development_mode.html#legacy-behavior) Instead, some strange-looking files starting with an__editable__prefix are added tosite-packages, which Python somehow uses to find the editable package.This PR looks for those
__editable__files in each entry insys.path, and then uses the_EditableFinderthingies insys.meta_pathto find the install location of the editable Python packages, and finally uses those locations when looking forjuliapkg.jsonfiles.