HS: Prune stale cache files from disk v3 #14270
Open
+489
−37
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 10 commits
November 4, 2025 12:50
To have a system-level overview of when was the last time the file was used, update the file modification timestamp to to the current time. This is needed to remove stale cache files of the system. Access time is not used as it may be, on the system level, disabled. Ticket: 7830
Ticket: 7830
Hyperscan MPM can cache the compiled contexts to files. This however grows as rulesets change and leads to bloating the system. This addition prunes the stale cache files based on their modified file timestamp. Part of this work incorporates new model for MPM cache stats to split it out from the cache save function and aggregate cache-related stats in one place (newly added pruning). Ticket: 7830
This is especially relevant for multi-instance simultaneous setups as we might risk read/write races.
lukashino
requested review from
a team,
jasonish,
jufajardini and
victorjulien
as code owners
Codecov Report❌ Patch coverage is Additional details and impacted files@@ Coverage Diff @@
## main #14270 +/- ##
==========================================
- Coverage 84.14% 84.14% -0.01%
==========================================
Files 1013 1013
Lines 262354 262574 +220
==========================================
+ Hits 220764 220930 +166
- Misses 41590 41644 +54
Flags with carried forward coverage won't be shown. Click here to find out more. 🚀 New features to boost your workflow:
|
|
Information: QA ran without warnings. Pipeline = 28312 |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Follow-up of #14212
Link to ticket: https://redmine.openinfosecfoundation.org/issues/7830
Describe changes:
v3:
v2:
v1:
The logic to determine a stale file is currently based on the modification timestamp in the file systems. The accessed time stamp was not used as it may be switched off. Alternatively, we could use a local DB/notekeeping file of the last used files/caches but this approach seemed simpler.
I can also add GitHub CI tests, I thought of some scenarios.