Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Change Conda channel order #3237

Merged
merged 6 commits into from
Dec 12, 2016
Merged

Change Conda channel order #3237

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
8519e12
change conda channel order
bgruening Dec 1, 2016
1f221aa
change channel order
bgruening Dec 1, 2016
0c91f65
Change channel order in documentation
bgruening Dec 1, 2016
8a80b2d
change order in galaxy.ini
bgruening Dec 12, 2016
3d49265
change oder in docs
bgruening Dec 12, 2016
f693643
change channel order in conda.py
bgruening Dec 12, 2016
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion config/galaxy.ini.sample
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -203,7 +203,8 @@ paste.app_factory = galaxy.web.buildapp:app_factory
# Pass debug flag to conda commands.
#conda_debug = False
# conda channels to enable by default (http://conda.pydata.org/docs/custom-channels.html)
#conda_ensure_channels = conda-forge,r,bioconda,iuc
# the recommended channel order is the one from BioConda (https://github.com/bioconda/bioconda-recipes/blob/master/config.yml#L8)
#conda_ensure_channels = iuc,bioconda,r,defaults,conda-forge
# Set to True to instruct Galaxy to look for and install missing tool
# dependencies before each job runs.
#conda_auto_install = False
Expand Down
2 changes: 1 addition & 1 deletion doc/source/admin/dependency_resolvers.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -182,7 +182,7 @@ debug
ensure_channels
conda channels to enable by default. See
http://conda.pydata.org/docs/custom-channels.html for more
information about channels. (default: conda-forge,r,bioconda,iuc).
information about channels. (default: iuc,bioconda,r,defaults,conda-forge).

auto_install
Set to True to instruct Galaxy to look for and install missing tool
Expand Down
2 changes: 1 addition & 1 deletion lib/galaxy/tools/deps/resolvers/conda.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@

DEFAULT_BASE_PATH_DIRECTORY = "_conda"
DEFAULT_CONDARC_OVERRIDE = "_condarc"
DEFAULT_ENSURE_CHANNELS = "conda-forge,r,bioconda,iuc"
DEFAULT_ENSURE_CHANNELS = "iuc,bioconda,r,defaults,conda-forge"
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry for being late to the game, but is it possible that this list should be reversed?
It appears that currently conda-forge takes the lowest priority.
Admittedly this is a bit confusing, but it appears that galaxyproject/tools-devteam#375 fails because bioconda takes the highest priority, while conda-forge should be highest. This also explains passing tests in bioconda but failing when we try to use them in galaxy.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to stick to the bioconda channel order and this was changed to give bioconda the opportunity to overwrite conda-forge. I don't agree with this move but it was decided this way and I think Galaxy should follow it. https://github.com/bioconda/bioconda-recipes/blob/master/config.yml#L11

Copy link
Member

@mvdbeek mvdbeek Jan 3, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We need to stick to the bioconda channel order

absolutely. but I don't think we do, given that for example for perl-gd the import tests pass (because the dependencies are pulled in from conda-forge, I presume). I don't completely understand the details yet, but to me it looks like galaxy is not respecting the channel order. Changing the order in planemo with --conda_channels has no effect whatsoever.

Copy link
Member

@mvdbeek mvdbeek Jan 3, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's possible that there are two issues:

  • For whatever reason the bioconda build system doesn't do what it's supposed to (either false positive test on the import test, or it does actually have conda-forge at a higher priority ... difficult to tell with the limited test output)
  • We're unable to change channel priority with planemo conda_install --conda_channels ...
    So I guess @nsoranzo is right to pin perl-gd to a higher libgd build number

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've noticed similar issues in my development environment when changing the channel order to pull in recipes I'm developing and uploading here https://anaconda.org/gregvonkuster/, but not related to planemo or galaxy.

The issue I've encountered is that if a package/version is is available in one of "iuc,bioconda,r,defaults,condo-forge", and I add an altered recipe for the package using the same version to my anaconda channel and update my channels to be "gregvonkuster,iuc,bioconda,r,defaults,condo-forge", the altered recipe is not pulled from my channel. Instead, the package is pulled from its original channel using the unchanged recipe.

This is all done from the command line without using planemo or galaxy. It seems that conda somehow is not accounting for my channel order.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When adding channels using conda config --add they should be added in reverse (from low to high priority). How are you adding these channels @gregvonkuster?
@mvdbeek do you have updated your bioconda-utils installation, is everything up to date?

Copy link
Member

@mvdbeek mvdbeek Jan 3, 2017 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I haven't used bioconda utils for this ... everything is done by travis, so that's whatever travis is using.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Note that the condarc is hard code in Planemo and even if you specify all the channel to change the order, Planemo just add those that are missing

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm running on Linux and using .condarc. I'm still running miniconda2, version 4.2.13. I have not tried the conda config --add channels approach, so I'll try it and see if it makes a difference. Thanks!

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Edit: it's within Galaxy


log = logging.getLogger(__name__)

Expand Down