Skip to content

Reduce cache size of CI caches #2336

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 24, 2025
Merged

Reduce cache size of CI caches #2336

merged 6 commits into from
Apr 24, 2025

Conversation

hosekpeter
Copy link
Contributor

@hosekpeter hosekpeter commented Apr 17, 2025

This pull request introduces a new workflow for cleaning up GitHub Actions caches and includes documentation to explain its purpose and usage. The workflow aims to manage cache growth by removing old and duplicate entries, improving efficiency and reducing disk usage.

Workflow Implementation:

  • Added a new workflow, .github/workflows/gh-cache-cleanup.yml, to automate GitHub Actions cache cleanup. It includes features such as scheduled weekly and monthly cleanups, cache deduplication, and a dry-run mode for safe testing.

Documentation:

  • Added docs/github-actions-cache-cleanup.md to describe the problem of cache growth, the solution provided by the new workflow, and detailed instructions for configuration, usage, and testing.Jira: XXXXX

Changes:

  • ...

@hosekpeter hosekpeter force-pushed the petr-hosek-PSGO-726 branch 10 times, most recently from e38ded8 to 17003d4 Compare April 17, 2025 12:32
@hosekpeter hosekpeter force-pushed the petr-hosek-PSGO-726 branch from 17003d4 to 12243fa Compare April 17, 2025 12:35
@hosekpeter hosekpeter changed the title Petr-hosek-PSGO-726 Reduce cache size of CI caches Apr 17, 2025
@hosekpeter
Copy link
Contributor Author

@hosekpeter hosekpeter marked this pull request as ready for review April 17, 2025 14:16
Copy link
Contributor

@Matovidlo Matovidlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For some reason we stopped using the S3 bucket, now we have GH cache which we do not need to clean at all it has it's own lifecycle.
This cleanup is needed only for S3 bucket this gh cache cleanup is done automatically

@hosekpeter
Copy link
Contributor Author

hosekpeter commented Apr 17, 2025

@Matovidlo I read somewhere the GitHub cache cleaner can't to remove the duplicate keys. You can set up it in the own workflow. It is not by default.

@hosekpeter hosekpeter marked this pull request as draft April 23, 2025 10:49
@hosekpeter hosekpeter force-pushed the petr-hosek-PSGO-726 branch 3 times, most recently from 7c7b4e6 to d92816d Compare April 23, 2025 10:59
@hosekpeter hosekpeter marked this pull request as ready for review April 23, 2025 11:38
@hosekpeter hosekpeter requested a review from Matovidlo April 23, 2025 17:12
Copy link
Contributor

@Matovidlo Matovidlo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. We need to take a look on caching using S3 as this cache is overloaded.
Typically it is 24-48 hours present

@hosekpeter hosekpeter merged commit 33f42e3 into main Apr 24, 2025
15 checks passed
@hosekpeter hosekpeter deleted the petr-hosek-PSGO-726 branch April 24, 2025 08:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants