Skip to content

Conversation

ebyhr
Copy link
Member

@ebyhr ebyhr commented Jan 15, 2025

Description

The following PR broke our integration with older REST catalogs:

AWS SDK bump broke our integration with S3-compatible storages (pre-2025 Minio, Vast, Dell EC etc):

Release notes

(x) This is not user-visible or is docs only, and no release notes are required.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this will break iceberg in storage like ECS. Can we force iceberg not to use storng integrity checks?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ebyhr
Copy link
Member Author

ebyhr commented Feb 13, 2025

I'm fixing product test failures.

@ebyhr ebyhr force-pushed the ebi/iceberg-1.8.0 branch from 849c3f5 to 98b1267 Compare February 15, 2025 11:40
@wendigo
Copy link
Contributor

wendigo commented Feb 15, 2025

@ebyhr do we plan to merge it? What's the scope of the strong integrity checks issue?

@ebyhr
Copy link
Member Author

ebyhr commented Feb 17, 2025

@wendigo I think we should wait for 1.8.1. There are 2 issues - REST catalog compatibility apache/iceberg#12294 and S3 strong integrity checks. The scope of S3 issue is limited because most IO happens with our file system module. However, the scope of REST catalog issue is unclear because it depends on each REST catalog's implementation. I believe this update breaks integration not just with Polaris and Unity, but with most REST catalogs.

@ebyhr
Copy link
Member Author

ebyhr commented Feb 20, 2025

Anther PR with 1.8.1 RC #25094

@ebyhr
Copy link
Member Author

ebyhr commented Feb 25, 2025

Closing as we will update to 1.8.1 instead #25094.

@ebyhr ebyhr closed this Feb 25, 2025
@ebyhr ebyhr deleted the ebi/iceberg-1.8.0 branch February 25, 2025 03:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

cla-signed iceberg Iceberg connector

Development

Successfully merging this pull request may close these issues.

2 participants