Skip to content

[controller][admin-tool] Add lookback time parameter to GetDeadStores API #1971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

MikeDafi
Copy link
Contributor

@MikeDafi MikeDafi commented Jul 29, 2025

Problem Statement

The current GetDeadStores API lacks the ability to specify a time window for dead store detection. In cases where the lookback window needs to be reduced such that the the store is considered dead recently but would have been considered NOT dead in the full lookback window.

Solution

Added an optional lookBackMS parameter to the GetDeadStores API using method overloading to maintain full backwards compatibility. The solution propagates this parameter through the entire stack:

  • DeadStoreStats interface: Added overloaded method with lookBackMS parameter
  • Admin interface: Added overloaded getDeadStores method
  • Controller implementations: VeniceHelixAdmin and VeniceParentHelixAdmin support new parameter
  • REST API: StoresRoutes handles optional look_back_ms query parameter
  • Client library: ControllerClient provides overloaded method
  • Admin tool: Added --look-back-ms CLI argument

Performance considerations: Zero overhead for existing calls as they use the original 3-parameter method. New 4-parameter method only incurs cost when explicitly used.

Testing: All Java compilation passes, method overloading maintains backwards compatibility.

Code changes

  • Added new code behind a config. If so list the config names and their default values in the PR description.
  • Introduced new log lines.
    • Confirmed if logs need to be rate limited to avoid excessive logging.

Concurrency-Specific Checks

Both reviewer and PR author to verify

  • Code has no race conditions or thread safety issues.
  • Proper synchronization mechanisms (e.g., `synchronized`, `RWLock`) are used where needed.
  • No blocking calls inside critical sections that could lead to deadlocks or performance degradation.
  • Verified thread-safe collections are used (e.g., `ConcurrentHashMap`, `CopyOnWriteArrayList`).
  • Validated proper exception handling in multi-threaded code to avoid silent thread termination.

How was this PR tested?

  • New unit tests added.
  • New integration tests added.
  • Modified or extended existing tests.
  • Verified backward compatibility (if applicable).

Does this PR introduce any user-facing or breaking changes?

  • No. You can skip the rest of this section.
  • Yes. Clearly explain the behavior change and its impact.
    "

@xunyin8
Copy link
Contributor

xunyin8 commented Jul 29, 2025

Typo in the description? "In cases where the lookback window needs to be reduced such that the the store is considered dead recently but would have been considered dead in the full lookback window." Are they not the same thing?

String clusterName,
String storeName,
boolean includeSystemStores,
long lookBackMS) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we foresee more parameters to be added in the future? Currently we already have includeSystemStores and lookBackMS. If we do foresee more parameters to be added in the future to simplify future changes and API compatibility we can use something like a param Map<String, String> similar to what we did in QueryParams

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Agreed, now using Map

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants