Skip to content

feat(frontend): support fragment level progress for snapshot backfill #22166

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: kwannoel/show-jobs-v3-3
Choose a base branch
from

Conversation

kwannoel
Copy link
Contributor

@kwannoel kwannoel commented Jun 9, 2025

I hereby agree to the terms of the RisingWave Labs, Inc. Contributor License Agreement.

What's changed and what's your intention?

This PR improves the backfill progress tracking for snapshot backfill by:

  1. Restructuring the backfill progress test files for better organization
  2. Adding support for tracking progress in snapshot backfill mode by:
    • Tracking the minimum epoch in backfill progress
    • Enhancing the rw_fragment_backfill_progress view to show 100% completion when the minimum epoch exceeds the backfill epoch
    • Adding is_snapshot_backfill and backfill_epoch fields to the table scan information
  3. Extending the test cases to verify both regular and snapshot backfill progress

These changes provide more accurate progress reporting for snapshot backfill operations, showing 100% completion when the backfill has processed all data up to the snapshot epoch.

Checklist

  • I have written necessary rustdoc comments.

Copy link
Contributor Author

kwannoel commented Jun 9, 2025

Warning

This pull request is not mergeable via GitHub because a downstack PR is open. Once all requirements are satisfied, merge this PR as a stack on Graphite.
Learn more

This stack of pull requests is managed by Graphite. Learn more about stacking.

@kwannoel kwannoel changed the title add epoch to backfill progress feat(frontend): support fragment level progress for snapshot backfill Jun 9, 2025
@github-actions github-actions bot added type/feature Type: New feature. and removed Invalid PR Title labels Jun 9, 2025
@kwannoel kwannoel marked this pull request as ready for review June 9, 2025 16:04
@kwannoel kwannoel requested a review from wenym1 June 9, 2025 16:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/feature Type: New feature.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant