What's the future of this project?

[cedricziel]

Hey there 👋

I'm writing a super small project (https://github.com/cedricziel/signaldb) and I'm in the process of adding iceberg support to it for some nice benefits over plain parquet.

I wonder how you imagine this project to continue? - Right now from what I can see, the differences between this implementation and the official iceberg-rust are quite big and I wonder what you have in mind for users of the library?

Do you think this project is going to be long-term evolving and maybe even eventually evolving into superseding the upstream iceberg-rust library? - You probably had good reasoning to introduce your own implementation of iceberg-rust and I'm curious about it.

It would be very helpful if you could add a bit of that intel to the readme as well.

Thanks a lot!

[JarroVGIT]

I have similar questions, I am considering adding Iceberg support to a (for now) private project, but I am unsure whether I should invest in the official iceberg-rust project or in this project. This project seems miles ahead and under more active development, but there is no official governance.

My main question is: are you counting on a merge of the official and this non-official implementation at some point? I understand there are no guarantees to be given, I am just trying to get a sense of how you see this evolving yourself.

[JanKaul]

Sorry for the late reply. Let me try to provide some background.

A long time ago I offered this implementation to the Apache project, before there was an official implementation (See the link to community sync video and the general discussion in the thread: apache/iceberg#5122 (comment)).

Later some folks from RisingWave and Databend started implementing iceberg in rust. We all got together to implement the official repo. At the time the decision was made to make the new implementation closer to their implementation and the Java implementation.

The projects are now quite different and some fundamentally different choices have been made. I don't have the time and motivation to port all the functionality from this crate over to the official implementation. Especially since I prefer the choices I've made and it will be a lot of work.

I understand that for users the stability of an Apache project is very important and I can't give you the same guarantees.

I hope this provides some insight.

[JarroVGIT]

Thanks for the response @JanKaul, really appreciate the background on how we got to the current situation. Really appreciate stipulating that out and it provided all the answers I required.

I just wanna say I really like your crates, nice work. I see there is some work to be done to be V3 compliant, are you open to PRs or is this something you would rather pursue yourself?

[JanKaul]

I'll gladly accept PRs. That would be really cool.

Maybe we can coordinate the work somehow. I also wanted to start on V3 eventually.

Are there any features that you are particularly interested in?