Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refine IpcSchemaEncoder API and docs #1

Merged

Conversation

alamb
Copy link

@alamb alamb commented Sep 24, 2024

Note this targets apache#6444

While reviewing the (very nice and very fast) work from @brancz I wanted to help improve the documentation and refine the API a little as part of the review. I figured it would be easier to propose these changes as a PR rather than comments

@CLAassistant
Copy link

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.
You have signed the CLA already but the status is still pending? Let us recheck it.

@alamb alamb marked this pull request as draft September 24, 2024 20:08
}

/// Specify a dictionary tracker to use
pub fn with_dictionary_tracker(
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since the lower level build_field takes an optional dictionary tracker it seems like we should permit that still

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It actually seems like many of the crate private APIs like build_field could be pulled into this IpcSchemaEncoder but I don't want to go too crazy now. Maybe some future PR

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah I was thinking the same, I wanted to keep the changes small

///
/// See also [`fb_to_schema`] for the reverse operation
///
/// # Example
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I always found these APIs confusing to use, so I figured I would make them easier to use here

pub fn schema_to_fb(schema: &Schema) -> FlatBufferBuilder<'_> {
let mut fbb = FlatBufferBuilder::new();
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here and below, I removed the duplicate implementation and call into the new encoder -- so we can keep the old API the same but the code is shared

@@ -208,8 +208,9 @@ impl IpcDataGenerator {
) -> EncodedData {
let mut fbb = FlatBufferBuilder::new();
let schema = {
let mut converter = IpcSchemaConverter::new(dictionary_tracker);
let fb = converter.schema_to_fb_offset(&mut fbb, schema);
let fb = IpcSchemaEncoder::new()
Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this usage looks a bit nicer

@alamb alamb marked this pull request as ready for review September 24, 2024 20:26
@brancz brancz merged commit 4d46fcb into polarsignals:ipc-set-dict-id Sep 25, 2024
1 check was pending
@alamb alamb deleted the alamb/improve_docs_comments branch September 25, 2024 16:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants