Skip to content

Block-builder-scheduler: avoid committing offsets for partitions that have had no consumption #12130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

seizethedave
Copy link
Contributor

@seizethedave seizethedave commented Jul 18, 2025

What this PR does

There was a bug introduced in #11867 where if a block-builder-scheduler encounters a partition in this state:

  • the partition has an end offset > 0
  • the block-builder consumer group has no commit for that partition

then:

  • the block-builder-scheduler would store a committed offset of 0 in its state

which leads to:

  1. the block-builder-scheduler would erroneously detect an offset gap the next time a job is created for that partition.
  2. the scheduler would emit a value of 0 for the partition in the cortex_blockbuilder_scheduler_partition_committed_offset gauge.

(This can happen if the partition was hasn't been written to since before the scheduler or its consumer group existed in that environment.)

To fix this, I am introducing a special state of offsetEmpty (-1) to the offsets that are handled specially.

Checklist

  • Tests updated.
  • (n/a) Documentation added.
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]. If changelog entry is not needed, please add the changelog-not-needed label to the PR.
  • (n/a) about-versioning.md updated with experimental features.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant