Make cache key configurable

[brunoalr]

Currently, as far as I understand, the action uses conan-${{ steps.install-conan.outputs.conan-version }}-${{ runner.os }} (see https://github.com/conan-io/setup-conan/blob/main/action.yml#L112) as cache key. It would be even better to make it configurable by the users in order to support uses cases such as multiple configurations for the OS (e.g. different compilers, debug vs release etc).

[memsharded]

It would be even better to make it configurable by the users in order to support uses cases such as multiple configurations for the OS (e.g. different compilers, debug vs release etc).

Could you please elaborate a bit further? The Conan cache already manages automatically different compilers, different configurations, architectures, OS, static/shared, and any other variation, so this seems unnecessary

[brunoalr]

Hi, @memsharded, thanks for the quick reply.

What happens is that GHA (github actions) caches are bound only to an os name (coming from ${{ runner.os }} and conan version. For example, if you have a matrix job that runs on both ubuntu 22.04 and ubuntu 24.04, the caches' keys will clash: both will be conan-2.7-Linux given that ${{ runner.os }} yields Linux for both images.

Because of this clash, only the build that finishes first will actually save the cache with key conan-2.7-Linux. In a subsequent run, there will be indeed GHA cache hits for both ubuntu versions but the contents will only be a proper conan cache hit for the one whose job finished first in the previous run.

I created a minimal repository to reproduce the issue: it contains a workflow that calls setup-conan with cache_packages: 'true' and a conanfile.txt with a recipe for protobuf/6.30.1.

First I deleted all GHA caches to ensure a clean build, then opened a PR. In the first run, there were no GHA caches for conan-2.17-Linux neither for conan-2.7-macOS as expected. Thus, the recipe had to be built from source and the ubuntu builds took around 5min each and macos builds took around 3min each. Also, in caches there were created only 2 entries: conan-2.17-Linux and conan-2.7-macOS.

Then I triggered a re-build. There were GHA cache hits for all 4 jobs, but only for ubuntu-24.04 and macos-15 they were proper conan cache hits. There were no recipe re-builds for ubuntu-24.04 (took around 20s) and macos-15 (took around 50s). But there were recipe re-builds for ubuntu-22.04(took around 5min) and for macos-14 (took around 4min).

[uilianries]

@brunoalr Salve 👋 E obrigado por sua detalhada mensagem!

Only to add a bit more details:

• Ubuntu 24.04 uses GCC-13 by default
• Ubuntu 20.04 uses GCC-11 by default
• When the GHA cache is recovered, it restores the Conan cache with only one of those packages (or GCC11 or GCC13, whichever was built first), resulting in a missing package.

From the GHA Cache, we can not re-use the same cache, or at least, the package folder, as the cache is immutable: https://github.com/actions/cache/blob/main/tips-and-workarounds.md#update-a-cache

Another alternative would be uploading the Conan cache: conan cache save + conan upload to a Github Artifact, but I see a few limitations when considering updating the saved cache by a concurrent CI job.

[uilianries]

I'll do a second POC, based on using cache from user side, instead of internalizing it in the Conan Action.

[brunoalr]

I'll do a second POC, based on using cache from user side, instead of internalizing it in the Conan Action.

Opa, salve! Tudo bom aí?

If you didn't do that yet, I had already experimented with it in a PR in my repo and it works as I expected. It's a bit messy, though (lots of push forces and commented out code since I am working by myself 😬).

[brunoalr]

IMO, it would be really nice if setup-conan had a similar (if not equal) set of inputs from https://github.com/hendrikmuhs/ccache-action.

If I were proficient with JS/TS I would have opened a PR myself 😅

[uilianries]

@brunoalr Hello again!

After doing more experiments and talking with the team, it seems your case is just the top of the iceberg, and it may cause more trouble than solving problems in the future.

When this action was created, it was a very simple proposal for installing Conan in GHA and nothing more. But then we added a few features, thinking about the most general scenarios, including caching builds, of course.

However, the action actions/cache is not well designed for those cases when you need to cache multiple jobs in parallel and then recover them later, it would require multiple caches, one for each job to keep a single cache key. Your case is much smaller, as you only need to reformulate the cache key. But, thinking about regular cases when using Artifactory or Conan Server, a common server is available for all packages to be downloaded. This scenario would be possible to mimic using actions/upload-artifact and then storing it in a cache using actions/cache, but it requires a lot of effort to maintain it later.

In summary, changing the option cache_packages or adding extra support for caching packages does not provide long-term benefits, as it will result in a snowball of issues.

My suggestion is to keep cache_packages: false (default) and manage your own build using actions/cache instead.

I made a workflow script trying to demonstrate a case with 4 different packages building in parallel, then recovering them later: test_cache.yml.txt (you can see it running here: https://github.com/uilianries/setup-conan/actions/runs/15731725430?pr=3)

It uses a common cache for each OS (the actions/cache does not work well for cross-OS use in this scenario), where those packages built previously are stored as artifacts, then pushed to a cache as a second step. Still, it lacks several features, like avoiding pushing a package that's cached already, but it's just to demonstrate how wild it can be quickly.

In the end, using the cache action may work for very simple cases, but the best option would be to use a Conan registry to keep your packages cached and well managed without pain.

Please, feel free to share your thoughts. Até logo!

[Rotonen]

Coming in from #8, what we seek is roughly along the lines of what I'm doing here for controlling uniqueness of the ccache GitHub Caches.

https://github.com/anura-engine/anura/blob/7d1b6c427bcd44dc92d1ba169fbf0f83bdf4e100/.github/workflows/pr-smoketest-debian.yaml#L65

Expose the key for advanced users to control in the with block and we're good and for the vast majority of users you need not care.

[Rotonen]

As an additional angle to consider: when you provide the set of paths to cache and a configurable cache key, we can trust that you know the internals of Conan and the rest of the community can stand on the shoulders of giants. And it's on us to make sure we use unique cache keys in build matrices as per my example above.

When we have to roll our own caching downstream we all need to independently look into how Conan actually works on a very detailed level and then also as a side effect incur a need to keep that inspection result up to date on our ends (just in case something in Conan changes /wrt how it stores its outputs, which would break caching). And this all happens as unique multiplicative effort across the community resulting in many unique solutions to the same problem over time with their own unique bugs and subtleties, which probably also bubble back up to Conan one way or another.

I'm assuming you dread a flood of confusing support cases around people reusing the cache key in a build matrix across a set of builds. Trivial solution: document this in the readme and keep tapping the sign.