Skip to content

db: incorporate blob files into EstimateDiskUsage #4621

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jbowens opened this issue Apr 24, 2025 · 0 comments
Open

db: incorporate blob files into EstimateDiskUsage #4621

jbowens opened this issue Apr 24, 2025 · 0 comments

Comments

@jbowens
Copy link
Collaborator

jbowens commented Apr 24, 2025

When estimating the disk usage between two bounds, we should incorporate contribution from blob files referenced from overlapping sstables. When there's partial overlap with a sstable, we don't have a means of accurately determining how much of a blob reference comes from the span passed into EstimateDiskUsage specifically. I suppose we could crudely estimate it using linear interpolation.

We use (*sstable.Reader).EstimateDiskUsage(start, end) to get an accurate estimate of the disk usage within the sstable itself. We divide that by the TableMetadata.Size to get an estimate of the fraction of the table's blob references that fall within the range. Then for each blob reference, we scale the uncompressd ValueLen by that fraction to get estimates of the uncompressed value quantities. Then we scale each of those by the compression ratio of the blob file and sum them all together.

Jira issue: PEBBLE-424

Epic CRDB-20379

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant