Skip to content

sstable: persist feature set in footer #4652

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
jbowens opened this issue Apr 29, 2025 · 0 comments
Open

sstable: persist feature set in footer #4652

jbowens opened this issue Apr 29, 2025 · 0 comments

Comments

@jbowens
Copy link
Collaborator

jbowens commented Apr 29, 2025

In #4557 we're adding a new type that concisely encodes the set of features used by a sstable. For now, this is derived from the Properties when we open a sstable. Ideally in the future we wouldn't need to read the properties block at all when we open a sstable.

When we open a sstable, the first thing we do is read the fixed-length footer at the very end of the file. See this comment for more context on the layout of a sstable. We should begin writing the uint32 feature set as a part of the footer. This is a backwards incompatible change, so we'll need to introduce a new TableFormat.

At write time we should compute the feature set from the properties we've derived, and then include it in the footer we write out (but only if the configured table format is sufficiently high).

At read time after we parse the table format, we'll extract the feature set only if the table format is sufficiently high. For now we can just plumb it up and then validate that the feature set we derive from the properties block matches the feature set we read from the footer.

Eventually we'll be able to avoid reading the properties block when constructing a new reader if the table has a sufficiently high table format.

Jira issue: PEBBLE-429

Jira issue: PEBBLE-435

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

2 participants