Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add IpcSchemaEncoder, deprecate ipc schema functions, Fix IPC not respecting not preserving dict ID #6444

Merged
merged 7 commits into from
Sep 25, 2024

Commits on Sep 23, 2024

  1. Configuration menu
    Copy the full SHA
    4edafeb View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. arrow-ipc: Always set dict ID in IPC from dictionary tracker

    This decouples dictionary IDs that end up in IPC from the schema further
    because the dictionary tracker always first gathers the dict ID for each
    field whether it is pre-defined and preserved or not.
    
    Then when actually writing the IPC bytes the dictionary ID is always
    taken from the dictionary tracker as opposed to falling back to the
    `Field` of the `Schema`.
    brancz committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    d21569e View commit details
    Browse the repository at this point in the history
  2. arrow-ipc: Read dictionary IDs from dictionary tracker in correct order

    When dictionary IDs are not preserved, then they are assigned depth
    first, however, when reading them from the dictionary tracker to write
    the IPC bytes, they were previously read from the dictionary tracker in
    the order that the schema is traversed (first come first serve), which
    caused an incorrect order of dictionaries serialized in IPC.
    brancz committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    200d1f5 View commit details
    Browse the repository at this point in the history
  3. Configuration menu
    Copy the full SHA
    9f2380c View commit details
    Browse the repository at this point in the history
  4. reduce repeated code

    alamb committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    6bc335e View commit details
    Browse the repository at this point in the history

Commits on Sep 25, 2024

  1. Merge pull request #1 from alamb/alamb/improve_docs_comments

    Refine IpcSchemaEncoder API and docs
    brancz authored Sep 25, 2024
    Configuration menu
    Copy the full SHA
    4d46fcb View commit details
    Browse the repository at this point in the history
  2. Fix lints

    brancz committed Sep 25, 2024
    Configuration menu
    Copy the full SHA
    41ca359 View commit details
    Browse the repository at this point in the history