Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add decoder for DELTA_BYTE_ARRAY to Parquet reader #14101

Merged
merged 88 commits into from
Nov 16, 2023
Merged

Add decoder for DELTA_BYTE_ARRAY to Parquet reader #14101

merged 88 commits into from
Nov 16, 2023

Commits on Sep 13, 2023

  1. add decoder for DELTA_BYTE_ARRAY

    @etseidl
    etseidl committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    1a6f842 View commit details
    Browse the repository at this point in the history

  2. switch to scoped enum

    @etseidl
    etseidl committed Sep 13, 2023
    Configuration menu
    Copy the full SHA
    d4a5ae5 View commit details
    Browse the repository at this point in the history

Commits on Sep 14, 2023

  1. Merge branch 'branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 14, 2023
    Configuration menu
    Copy the full SHA
    e83c281 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 14, 2023
    Configuration menu
    Copy the full SHA
    050d0c6 View commit details
    Browse the repository at this point in the history

Commits on Sep 15, 2023

  1. Merge branch 'rapidsai:branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 15, 2023
    Configuration menu
    Copy the full SHA
    8bf69a3 View commit details
    Browse the repository at this point in the history

Commits on Sep 18, 2023

  1. Merge branch 'rapidsai:branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 18, 2023
    Configuration menu
    Copy the full SHA
    54fecc6 View commit details
    Browse the repository at this point in the history

Commits on Sep 19, 2023

  1. Merge branch 'rapidsai:branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 19, 2023
    Configuration menu
    Copy the full SHA
    dab0d40 View commit details
    Browse the repository at this point in the history

Commits on Sep 20, 2023

  1. need to copy string data on default stream

    @etseidl
    etseidl committed Sep 20, 2023
    Configuration menu
    Copy the full SHA
    25a42eb View commit details
    Browse the repository at this point in the history

  2. Merge branch 'rapidsai:branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 20, 2023
    Configuration menu
    Copy the full SHA
    15167b2 View commit details
    Browse the repository at this point in the history

Commits on Sep 21, 2023

  1. Merge branch 'branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 21, 2023
    Configuration menu
    Copy the full SHA
    fac8c0b View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2023

  1. Merge branch 'rapidsai:branch-23.10' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 27, 2023
    Configuration menu
    Copy the full SHA
    36619e3 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 27, 2023
    Configuration menu
    Copy the full SHA
    b8dcbb5 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 27, 2023
    Configuration menu
    Copy the full SHA
    46d8ea2 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 27, 2023
    Configuration menu
    Copy the full SHA
    edda85b View commit details
    Browse the repository at this point in the history

Commits on Sep 28, 2023

  1. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    c207433 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    ce79d18 View commit details
    Browse the repository at this point in the history

  3. set error code in delta byte array decoder

    @etseidl
    etseidl committed Sep 28, 2023
    Configuration menu
    Copy the full SHA
    9b2a67e View commit details
    Browse the repository at this point in the history

Commits on Sep 29, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Sep 29, 2023
    Configuration menu
    Copy the full SHA
    af5fcf6 View commit details
    Browse the repository at this point in the history

  2. redo BitAnd and BitOr

    @etseidl
    etseidl committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    20dc29f View commit details
    Browse the repository at this point in the history

  3. Merge branch 'decode_delta_byte_arr' of github.com:etseidl/cudf into … 

    …decode_delta_byte_arr
    @etseidl
    etseidl committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    a638fd7 View commit details
    Browse the repository at this point in the history

  4. use snake case for names

    @etseidl
    etseidl committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    cba2858 View commit details
    Browse the repository at this point in the history

  5. add test

    @etseidl
    etseidl committed Sep 29, 2023
    Configuration menu
    Copy the full SHA
    c52481e View commit details
    Browse the repository at this point in the history

Commits on Oct 2, 2023

  1. cleanups

    @etseidl
    etseidl committed Oct 2, 2023
    Configuration menu
    Copy the full SHA
    5a1da6b View commit details
    Browse the repository at this point in the history

Commits on Oct 3, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 3, 2023
    Configuration menu
    Copy the full SHA
    1dd1d7f View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 3, 2023
    Configuration menu
    Copy the full SHA
    bee540e View commit details
    Browse the repository at this point in the history

Commits on Oct 4, 2023

  1. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    fad139c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    d7cf17a View commit details
    Browse the repository at this point in the history

  3. add some documentation

    @etseidl
    etseidl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    af9fb24 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 4, 2023
    Configuration menu
    Copy the full SHA
    63e3740 View commit details
    Browse the repository at this point in the history

  5. more cleanup

    @etseidl
    etseidl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    c8e3938 View commit details
    Browse the repository at this point in the history

  6. more cleanup

    @etseidl
    etseidl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    0332923 View commit details
    Browse the repository at this point in the history

  7. a little more cleanup

    @etseidl
    etseidl committed Oct 4, 2023
    Configuration menu
    Copy the full SHA
    f5f2bd5 View commit details
    Browse the repository at this point in the history

Commits on Oct 6, 2023

  1. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 6, 2023
    Configuration menu
    Copy the full SHA
    3f66cd8 View commit details
    Browse the repository at this point in the history

Commits on Oct 9, 2023

  1. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    5f2a7d4 View commit details
    Browse the repository at this point in the history

  2. clean up some docstrings and move where str_bytes is initialized

    @etseidl
    etseidl committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    2c30646 View commit details
    Browse the repository at this point in the history

  3. revert move of str_bytes reset...broke chunked reads

    @etseidl
    etseidl committed Oct 9, 2023
    Configuration menu
    Copy the full SHA
    c2dbb47 View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 10, 2023
    Configuration menu
    Copy the full SHA
    d0b01d0 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2023

  1. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 11, 2023
    Configuration menu
    Copy the full SHA
    b24ec79 View commit details
    Browse the repository at this point in the history

Commits on Oct 16, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 16, 2023
    Configuration menu
    Copy the full SHA
    a2ed863 View commit details
    Browse the repository at this point in the history

Commits on Oct 18, 2023

  1. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    7ab1e1a View commit details
    Browse the repository at this point in the history

  2. remove redundant comment

    @etseidl
    etseidl committed Oct 18, 2023
    Configuration menu
    Copy the full SHA
    71b979c View commit details
    Browse the repository at this point in the history

Commits on Oct 20, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 20, 2023
    Configuration menu
    Copy the full SHA
    5ce72a4 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    cb15ace View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    c3ea439 View commit details
    Browse the repository at this point in the history

  4. fix test for supported encodings lost in merge

    @etseidl
    etseidl committed Oct 20, 2023
    Configuration menu
    Copy the full SHA
    46d179a View commit details
    Browse the repository at this point in the history

  5. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @vuule
    vuule authored Oct 20, 2023
    Configuration menu
    Copy the full SHA
    c898194 View commit details
    Browse the repository at this point in the history

Commits on Oct 26, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 26, 2023
    Configuration menu
    Copy the full SHA
    cf356c4 View commit details
    Browse the repository at this point in the history

Commits on Oct 27, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 27, 2023
    Configuration menu
    Copy the full SHA
    f30a5c8 View commit details
    Browse the repository at this point in the history

Commits on Oct 30, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Oct 30, 2023
    Configuration menu
    Copy the full SHA
    ddacf38 View commit details
    Browse the repository at this point in the history

  2. move description of DELTA_BYTE_ARRAY to where it belongs. add some 

    clarification and rename suffix_data.
    @etseidl
    etseidl committed Oct 30, 2023
    Configuration menu
    Copy the full SHA
    29df5bd View commit details
    Browse the repository at this point in the history

Commits on Nov 1, 2023

  1. typo caught by review 

    Co-authored-by: Vukasin Milovanovic <[email protected]>
    @etseidl @vuule
    etseidl and vuule authored Nov 1, 2023
    Configuration menu
    Copy the full SHA
    21d7f9b View commit details
    Browse the repository at this point in the history

  2. update calculation of temp_bytes

    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    ebef98d View commit details
    Browse the repository at this point in the history

  3. Merge branch 'decode_delta_byte_arr' of github.com:etseidl/cudf into … 

    …decode_delta_byte_arr
    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    3f2b26e View commit details
    Browse the repository at this point in the history

  4. fix comment 

    Co-authored-by: Vukasin Milovanovic <[email protected]>
    @etseidl @vuule
    etseidl and vuule authored Nov 1, 2023
    Configuration menu
    Copy the full SHA
    cb2d7ce View commit details
    Browse the repository at this point in the history

  5. add some explanation for what is going on in ComputePageStringSizes

    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    5c1549b View commit details
    Browse the repository at this point in the history

  6. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 1, 2023
    Configuration menu
    Copy the full SHA
    e508e3a View commit details
    Browse the repository at this point in the history

  7. use device_uvector

    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    bad9f6a View commit details
    Browse the repository at this point in the history

  8. implement a TODO

    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    75e23fc View commit details
    Browse the repository at this point in the history

  9. use exec_policy_nosync when calculating string sizs

    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    300f492 View commit details
    Browse the repository at this point in the history

  10. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 1, 2023
    Configuration menu
    Copy the full SHA
    9dff818 View commit details
    Browse the repository at this point in the history

  11. Merge branch 'decode_delta_byte_arr' of github.com:etseidl/cudf into … 

    …decode_delta_byte_arr
    @etseidl
    etseidl committed Nov 1, 2023
    Configuration menu
    Copy the full SHA
    3a0099b View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 7, 2023
    Configuration menu
    Copy the full SHA
    e618ad0 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    cd15adf View commit details
    Browse the repository at this point in the history

  3. fix typo

    @etseidl
    etseidl committed Nov 7, 2023
    Configuration menu
    Copy the full SHA
    fe7e5a2 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2023

  1. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    9556bc2 View commit details
    Browse the repository at this point in the history

  2. add delta binary roundtrip test

    @etseidl
    etseidl committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    e9baf16 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    59c9ef1 View commit details
    Browse the repository at this point in the history

  4. test a larger range of rows around critical numbers for the delta dec… 

    …oder
    @etseidl
    etseidl committed Nov 8, 2023
    Configuration menu
    Copy the full SHA
    4e1160a View commit details
    Browse the repository at this point in the history

  5. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 8, 2023
    Configuration menu
    Copy the full SHA
    4affd2f View commit details
    Browse the repository at this point in the history

Commits on Nov 9, 2023

  1. add struct of list test for delta

    @etseidl
    etseidl committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    734023b View commit details
    Browse the repository at this point in the history

  2. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 9, 2023
    Configuration menu
    Copy the full SHA
    c39990e View commit details
    Browse the repository at this point in the history

  3. future proof test by adding param for string encoding

    @etseidl
    etseidl committed Nov 9, 2023
    Configuration menu
    Copy the full SHA
    8b6ed5e View commit details
    Browse the repository at this point in the history

Commits on Nov 11, 2023

  1. Merge branch 'branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 11, 2023
    Configuration menu
    Copy the full SHA
    42c960f View commit details
    Browse the repository at this point in the history

Commits on Nov 13, 2023

  1. fix some comments

    @etseidl
    etseidl committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    5e4b2dd View commit details
    Browse the repository at this point in the history

  2. a few fixes suggested in review

    @etseidl
    etseidl committed Nov 13, 2023
    Configuration menu
    Copy the full SHA
    8338207 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 13, 2023
    Configuration menu
    Copy the full SHA
    40dbd59 View commit details
    Browse the repository at this point in the history

Commits on Nov 14, 2023

  1. Merge branch 'rapidsai:branch-23.12' into decode_delta_byte_arr

    @etseidl
    etseidl authored Nov 14, 2023
    Configuration menu
    Copy the full SHA
    982285c View commit details
    Browse the repository at this point in the history

  2. add sanity check to failing test

    @etseidl
    etseidl committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    8bf6edc View commit details
    Browse the repository at this point in the history

  3. fix overrun caused by large mini-blocks

    @etseidl
    etseidl committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    ea4c283 View commit details
    Browse the repository at this point in the history

  4. check that the mini-block size is not too large to handle

    @etseidl
    etseidl committed Nov 14, 2023
    Configuration menu
    Copy the full SHA
    cf5abe5 View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2023

  1. smack redundant statement 

    Co-authored-by: nvdbaranec <[email protected]>
    @etseidl @nvdbaranec
    etseidl and nvdbaranec authored Nov 15, 2023
    Configuration menu
    Copy the full SHA
    b697cb8 View commit details
    Browse the repository at this point in the history

  2. add const 

    Co-authored-by: nvdbaranec <[email protected]>
    @etseidl @nvdbaranec
    etseidl and nvdbaranec authored Nov 15, 2023
    Configuration menu
    Copy the full SHA
    07b2f79 View commit details
    Browse the repository at this point in the history

  3. Merge branch 'decode_delta_byte_arr' of github.com:etseidl/cudf into … 

    …decode_delta_byte_arr
    @etseidl
    etseidl committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    ea9eac0 View commit details
    Browse the repository at this point in the history

  4. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    2004caa View commit details
    Browse the repository at this point in the history

  5. remove some unnecessary syncthreads calls

    @etseidl
    etseidl committed Nov 15, 2023
    Configuration menu
    Copy the full SHA
    883a236 View commit details
    Browse the repository at this point in the history

Commits on Nov 16, 2023

  1. Revert "remove some unnecessary syncthreads calls" 

    This reverts commit 883a236.
    @etseidl
    etseidl committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    0844d8e View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/branch-23.12' into decode_delta_… 

    …byte_arr
    @etseidl
    etseidl committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    7a2e546 View commit details
    Browse the repository at this point in the history

  3. add a TODO to remove some syncthreads calls

    @etseidl
    etseidl committed Nov 16, 2023
    Configuration menu
    Copy the full SHA
    2f83f8a View commit details
    Browse the repository at this point in the history