Skip to content

Conversation

@daler
Copy link
Member

@daler daler commented May 21, 2025

#959 uses ghcr.io as a cache mechanism. However, images stored there are not straightforward to clean up. Artifacts, on the other hand, expire after 90 days. This PR is an experiment to see if it will simplify things further to use artifacts rather than ghcr.io.

@daler daler changed the base branch from master to simplify-unify-containers May 21, 2025 21:39
@daler daler force-pushed the simplify-unify-containers-artifacts branch from 8476e18 to 4716da3 Compare May 21, 2025 22:03
@daler daler force-pushed the simplify-unify-containers-artifacts branch from 1b04f59 to 2fc00f0 Compare June 1, 2025 19:18
@daler
Copy link
Member Author

daler commented Jun 1, 2025

Notes so far:

  • Artifacts are created out of containers created during each job and then downloaded if/when needed in subsequent jobs. podman load <tarball> then is run on the downloaded artifacts. This seems straightforward, and I think this is a little more transparent than the previous "ghcr.io as a cache" strategy
  • I removed caching and dependencies. Every job runs every time. This is terribly inefficient for debugging! But it does reduce complexity, which is hopefully more important for maintainability over the long term.
  • Images are created in podman but are used in docker by bioconda-utils. Previously, pushing to ghcr.io made this simpler because both docker and podman could push/pull to the registry in a runtime-agnostic manner. Now, passing artifacts around makes things more annoying especially with respect to manifests.
  • Apparently manifests cannot be used with a local docker runtime and MUST be pushed to a registry for docker to use them. So this PR adds a local docker registry for the CI running on localhost:5000.
  • Where things currently break is in the manifest creation. The idea is for buildah to build a manifest out of the existing archs and push it to the local docker registry at localhost:5000, and then get bioconda-utils to use that registry for the tests.

@daler daler force-pushed the simplify-unify-containers-artifacts branch from 04030d8 to 49b1b7c Compare June 3, 2025 21:56
@daler daler force-pushed the simplify-unify-containers-artifacts branch from 8294e6c to 279a0b7 Compare June 4, 2025 12:16
@daler daler force-pushed the simplify-unify-containers-artifacts branch from 9b5fa62 to 99507f7 Compare June 5, 2025 14:37
@daler
Copy link
Member Author

daler commented Jun 9, 2025

In good shape; closing in favor of the cleaned-up #1049.

@daler daler closed this Jun 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants