Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature]Parquet version upgrade to support LZ4_RAW compression format #7223

Open
2 of 3 tasks
liunaijie opened this issue Jul 17, 2024 · 4 comments · May be fixed by #7939
Open
2 of 3 tasks

[Feature]Parquet version upgrade to support LZ4_RAW compression format #7223

liunaijie opened this issue Jul 17, 2024 · 4 comments · May be fixed by #7939
Assignees
@liunaijie
Labels
help wanted waiting for review

Comments

@liunaijie
Copy link
Member

liunaijie commented Jul 17, 2024 •
edited
Loading

Search before asking

  • I had searched in the feature and found no similar feature requirement.

Description

Now, we use parquet-avro 1.12.3, in this version it not support read LZ4_RAW format file.
if we want support this format, we at least upgrade it to 1.13.0.

https://github.com/apache/parquet-java/blob/apache-parquet-1.12.3/parquet-common/src/main/java/org/apache/parquet/hadoop/metadata/CompressionCodecName.java

https://github.com/apache/parquet-java/blob/apache-parquet-1.13.0/parquet-common/src/main/java/org/apache/parquet/hadoop/metadata/CompressionCodecName.java

Usage Scenario

No response

Related issues

No response

Are you willing to submit a PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@liunaijie
Copy link
Member Author

image

also need add more data type support, like UINT8, UINT16, UINT32

@github-actions GitHub Actions
Copy link

This issue has been automatically marked as stale because it has not had recent activity for 30 days. It will be closed in next 7 days if no further activity occurs.

@github-actions github-actions bot added the stale label Aug 23, 2024
@github-actions GitHub Actions
Copy link

This issue has been closed because it has not received response for too long time. You could reopen it if you encountered similar problems in the future.

@fcb-xiaobo
Copy link
Contributor

please assign to me

@liunaijie liunaijie reopened this Aug 30, 2024
@fcb-xiaobo fcb-xiaobo removed their assignment Aug 30, 2024
@liunaijie liunaijie assigned liunaijie and unassigned fcb-xiaobo Oct 29, 2024
@liunaijie liunaijie linked a pull request Oct 29, 2024 that will close this issue
Draft
4 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@liunaijie liunaijie

Labels
help wanted waiting for review
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[improve] upgrade parquet version to 1.13.0 liunaijie/seatunnel
3 participants
@liunaijie @Hisoka-X @fcb-xiaobo