feat: add time based eviction to data managed by cachinglayer #43490

zhengbuqian · 2025-07-22T07:26:48Z

issue: #41435

also added disk capacity protection

sre-ci-robot · 2025-07-22T08:27:57Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: zhengbuqian

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~configs/OWNERS~~ [zhengbuqian]
~~internal/core/OWNERS~~ [zhengbuqian]
~~internal/querynodev2/OWNERS~~ [zhengbuqian]
~~pkg/util/OWNERS~~ [zhengbuqian]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

mergify · 2025-07-22T09:02:19Z

@zhengbuqian cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2025-07-22T09:02:20Z

@zhengbuqian go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2025-07-22T09:02:20Z

@zhengbuqian cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

zhengbuqian · 2025-07-22T09:06:47Z

rerun cpp-unit-test

zhengbuqian · 2025-07-22T09:06:51Z

rerun go-sdk

zhengbuqian · 2025-07-22T09:06:56Z

/run-cpu-e2e

zhengbuqian · 2025-07-22T09:08:26Z

rerun go-sdk

codecov · 2025-07-22T11:00:34Z

Codecov Report

Attention: Patch coverage is 67.54386% with 37 lines in your changes missing coverage. Please review.

Project coverage is 78.94%. Comparing base (990a25e) to head (1a52126).
Report is 19 commits behind head on master.

Files with missing lines	Patch %	Lines
internal/core/src/cachinglayer/lrucache/DList.cpp	69.11%	21 Missing ⚠️
internal/core/src/cachinglayer/Utils.cpp	45.83%	13 Missing ⚠️
internal/core/src/cachinglayer/Utils.h	66.66%	1 Missing ⚠️
internal/core/src/segcore/segcore_init_c.cpp	0.00%	1 Missing ⚠️
pkg/util/paramtable/component_param.go	91.66%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##           master   #43490      +/-   ##
==========================================
- Coverage   78.94%   78.94%   -0.01%     
==========================================
  Files        1566     1567       +1     
  Lines      224453   224778     +325     
==========================================
+ Hits       177205   177456     +251     
- Misses      40834    40896      +62     
- Partials     6414     6426      +12

Components	Coverage Δ
Client	`79.47% <ø> (ø)`
Core	`73.97% <62.88%> (+0.09%)`	⬆️
Go	`79.90% <94.11%> (-0.02%)`	⬇️

Files with missing lines	Coverage Δ
internal/core/src/cachinglayer/Manager.cpp	`83.33% <ø> (ø)`
internal/core/src/cachinglayer/lrucache/DList.h	`80.95% <ø> (ø)`
...ternal/core/src/cachinglayer/lrucache/ListNode.cpp	`82.40% <100.00%> (-0.14%)`	⬇️
internal/core/src/cachinglayer/lrucache/ListNode.h	`100.00% <ø> (ø)`
internal/querynodev2/server.go	`61.14% <100.00%> (+0.23%)`	⬆️
internal/core/src/cachinglayer/Utils.h	`71.85% <66.66%> (+0.28%)`	⬆️
internal/core/src/segcore/segcore_init_c.cpp	`21.34% <0.00%> (-0.25%)`	⬇️
pkg/util/paramtable/component_param.go	`98.16% <91.66%> (-0.02%)`	⬇️
internal/core/src/cachinglayer/Utils.cpp	`64.66% <45.83%> (-4.91%)`	⬇️
internal/core/src/cachinglayer/lrucache/DList.cpp	`79.81% <69.11%> (-3.74%)`	⬇️

... and 72 files with indirect coverage changes

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

mergify · 2025-07-22T11:23:07Z

@zhengbuqian cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify · 2025-07-22T11:25:08Z

@zhengbuqian cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2025-07-22T11:29:00Z

@zhengbuqian go-sdk check failed, comment rerun go-sdk can trigger the job again.

zhengbuqian · 2025-07-23T05:09:48Z

rerun go-sdk

zhengbuqian · 2025-07-23T05:09:52Z

/run-cpu-e2e

zhengbuqian · 2025-07-23T05:09:58Z

rerun cpp-unit-test

mergify · 2025-07-23T05:23:20Z

@zhengbuqian go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2025-07-23T05:23:21Z

@zhengbuqian cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify · 2025-07-23T05:25:17Z

@zhengbuqian cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2025-07-23T07:02:40Z

@zhengbuqian cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

Signed-off-by: Buqian Zheng <[email protected]>

mergify · 2025-07-23T19:03:48Z

@zhengbuqian go-sdk check failed, comment rerun go-sdk can trigger the job again.

mergify · 2025-07-23T19:09:24Z

@zhengbuqian cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2025-07-23T19:12:31Z

@zhengbuqian cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

zhengbuqian · 2025-07-24T02:32:31Z

rerun cpp-unit-test

zhengbuqian · 2025-07-24T02:32:38Z

/run-cpu-e2e

zhengbuqian · 2025-07-24T02:32:45Z

rerun go-sdk

mergify · 2025-07-24T03:37:00Z

@zhengbuqian cpu-e2e job failed, comment /run-cpu-e2e can trigger the job again.

mergify · 2025-07-24T03:50:17Z

@zhengbuqian cpp-unit-test check failed, comment rerun cpp-unit-test can trigger the job again.

mergify · 2025-07-24T04:36:38Z

@zhengbuqian go-sdk check failed, comment rerun go-sdk can trigger the job again.

sparknack · 2025-07-25T08:02:41Z

pkg/util/paramtable/component_param.go

@@ -2842,6 +2842,7 @@ type queryNodeConfig struct {
 	TieredCacheTouchWindowMs       ParamItem `refreshable:"false"`
 	TieredEvictionIntervalMs       ParamItem `refreshable:"false"`
 	TieredLoadingMemoryFactor      ParamItem `refreshable:"false"`
+	CacheCellUnaccssedSurvivalTime ParamItem `refreshable:"false"`


one typo: CacheCellUnaccssedSurvivalTime -> CacheCellUnaccessedSurvivalTime

sparknack · 2025-07-25T08:17:30Z

internal/core/src/cachinglayer/lrucache/DList.cpp

            // Combine with logical eviction target (take the maximum)
-            eviction_target.memory_bytes = std::max(
-                eviction_target.memory_bytes, physical_eviction_needed);
+            eviction_target.memory_bytes =


should eviction_target.file_bytes and min_eviction.file_bytes be updated here?

absolutely!

sparknack · 2025-07-25T08:25:24Z

internal/core/src/cachinglayer/lrucache/DList.cpp

            // we only need to evict for logical limit and we have succeeded.
            break;
        }

-        if (physical_eviction_needed = checkPhysicalMemoryLimit(size);
-            physical_eviction_needed == 0) {
+        if (physical_eviction_needed = checkPhysicalResourceLimit(size);


Since physical memory usage fluctuates over time, should min_eviction be set as a multiple of physical_eviction_needed?
This could handle the case where the minimum amount of evicted memory is immediately reused by another process outside the caching layer's control.

in checkPhysicalResourceLimit we already multiplied size by the loading factor

sparknack · 2025-07-25T08:29:08Z

internal/core/src/cachinglayer/lrucache/DList.cpp

@@ -144,12 +145,12 @@ DList::reserveMemoryInternal(const ResourceUsage& size) {
            "still need to evict {}",
            size.ToString(),
            evicted_size.ToString(),
-            FormatBytes(physical_eviction_needed));
+            physical_eviction_needed.ToString());
    }

    // Reserve resources (both checks passed)
    used_memory_ += size;


After introducing disk resource reservation, used_memory_ seems confusing. What about used_resource_ or another appropriate name?

sounds fair, done

sparknack · 2025-07-25T08:52:58Z

internal/core/src/cachinglayer/lrucache/DList.cpp

@@ -269,7 +269,8 @@ DList::tryEvict(const ResourceUsage& expected_eviction,
    ResourceUsage actively_pinned{0, 0};

    // accumulate victims using expected_eviction.
-    for (auto it = tail_; it != nullptr; it = it->next_) {
+    ListNode* it = nullptr;
+    for (it = tail_; it != nullptr; it = it->next_) {
        if (!would_help(it->size())) {


should these skipped it be considered expired items?

we don't know, they may be pinned, or not pinned.

why?

Signed-off-by: Buqian Zheng <[email protected]>

sre-ci-robot added approved size/L Denotes a PR that changes 100-499 lines. labels Jul 22, 2025

sre-ci-robot requested review from aoiasd and bigsheeper July 22, 2025 07:27

mergify bot added dco-passed DCO check passed. kind/feature Issues related to feature request from users labels Jul 22, 2025

buqian-zilliz force-pushed the time-based_eviction branch from 7714da9 to 41f5a6e Compare July 22, 2025 08:27

buqian-zilliz force-pushed the time-based_eviction branch from 41f5a6e to bf89009 Compare July 23, 2025 05:13

buqian-zilliz force-pushed the time-based_eviction branch from bf89009 to 5a843d3 Compare July 23, 2025 05:37

zhengbuqian added 2 commits July 24, 2025 01:59

added global time based eviction

4259a5b

Signed-off-by: Buqian Zheng <[email protected]>

added disk protection

5ddb0db

Signed-off-by: Buqian Zheng <[email protected]>

zhengbuqian force-pushed the time-based_eviction branch from 5a843d3 to 5ddb0db Compare July 23, 2025 17:59

sparknack reviewed Jul 25, 2025

View reviewed changes

address comments

1a52126

Signed-off-by: Buqian Zheng <[email protected]>

mergify bot added the ci-passed label Jul 25, 2025

feat: add time based eviction to data managed by cachinglayer #43490

Are you sure you want to change the base?

feat: add time based eviction to data managed by cachinglayer #43490

Conversation

zhengbuqian commented Jul 22, 2025

Uh oh!

sre-ci-robot commented Jul 22, 2025

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

zhengbuqian commented Jul 22, 2025

Uh oh!

zhengbuqian commented Jul 22, 2025

Uh oh!

zhengbuqian commented Jul 22, 2025

Uh oh!

zhengbuqian commented Jul 22, 2025

Uh oh!

codecov bot commented Jul 22, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

mergify bot commented Jul 22, 2025

Uh oh!

zhengbuqian commented Jul 23, 2025

Uh oh!

zhengbuqian commented Jul 23, 2025

Uh oh!

zhengbuqian commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

mergify bot commented Jul 23, 2025

Uh oh!

zhengbuqian commented Jul 24, 2025

Uh oh!

zhengbuqian commented Jul 24, 2025

Uh oh!

zhengbuqian commented Jul 24, 2025

Uh oh!

mergify bot commented Jul 24, 2025

Uh oh!

mergify bot commented Jul 24, 2025

Uh oh!

mergify bot commented Jul 24, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sparknack Jul 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

codecov bot commented Jul 22, 2025 •

edited

Loading

sparknack Jul 25, 2025 •

edited

Loading