You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Variant encoding looks like it's becoming part of the Parquet spec, and it'd be good to have Variant support in the Go library.
It looks like implementation of Variants is already under way in the Java Parquet library, as well as in Iceberg, so the spec seems to be settled enough that there shouldn't be massive changes in the base encoding. Looks like shredding still hasn't been tackled yet by any library, so that may be more in flux.
Variants will be a pretty powerful addon here for landing stuff in Iceberg- they generally have a good performance upside vs. using JSON or something similar.
Component(s)
Parquet
The text was updated successfully, but these errors were encountered:
Describe the enhancement requested
Variant encoding looks like it's becoming part of the Parquet spec, and it'd be good to have Variant support in the Go library.
It looks like implementation of Variants is already under way in the Java Parquet library, as well as in Iceberg, so the spec seems to be settled enough that there shouldn't be massive changes in the base encoding. Looks like shredding still hasn't been tackled yet by any library, so that may be more in flux.
Variants will be a pretty powerful addon here for landing stuff in Iceberg- they generally have a good performance upside vs. using JSON or something similar.
Component(s)
Parquet
The text was updated successfully, but these errors were encountered: