Skip to content

blob: add support for a 'long attribute' stored alongside handle #4588

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jbowens opened this issue Apr 18, 2025 · 3 comments
Open

blob: add support for a 'long attribute' stored alongside handle #4588

jbowens opened this issue Apr 18, 2025 · 3 comments

Comments

@jbowens
Copy link
Collaborator

jbowens commented Apr 18, 2025

With value separation, the cost of retrieving a value increases—an iterator may need to perform two additional block loads (one for the blob file index and one for the blob value block)

In CockroachDB, uncertainty interval checks require reading KVs' "local timestamp". This timestamp is encoded within the key-value pair's value (specifically as a field the MVCCValue protocol buffer). In mostsome cases, CockroachDB has no other need for the value other than retrieving this local timestamp.

To avoid retrieving the value just for the local timestamp, we'd like to add support for allowing a user to specify a LongAttribute that should be encoded alongside/within the inline blob handle when a value is separated. Then we should extend the LazyValue type to allow extraction of the LongAttribute from the blob handle suffix.

See the ShortAttribute type for an analogous concept that allows the user to stuff a few bits derived from the value into the block.ValuePrefix when a value is separated.

There's some prior art in @sumeerbhola's value separation prototype: master...sumeerbhola:pebble:blobt2#diff-536ef24c119680fc300327ac83776a1a1672559ca824078340345d8151a0074cR247-R267

Jira issue: PEBBLE-417

Epic CRDB-20379

@RaduBerinde
Copy link
Member

This would also be very useful for tiering. SQL would put the tiering value (timestamp) separately in roachpb.Value.

@RaduBerinde
Copy link
Member

RaduBerinde commented Apr 23, 2025

Ideally the local timestamp should become a different column rather than being part of the blob handle. We should think of a general way of mapping MVCCValue fields to columns instead of proto-encoding it.

@jbowens
Copy link
Collaborator Author

jbowens commented Apr 24, 2025

Yeah, maybe the caller should specify a colblk.KVSchema instead of a colblk.KeySchema and be allowed to specify which columns are guaranteed in-place. We don't have a need for it today, but if we do #4483 we can encode non-in-place columns within a blob block. This might eventually be useful for predicate pushdown.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants