Scope of minimal implementation

[paleolimbot]

I scraped together all the Java I remembered to put together an implementation for a GeoArrow data type (i.e., a representation of all the memory layouts currently in the spec). I also made a stub of ArrayData (a data type and a collection of buffers).

Other than those, I am not entirely sure of the scope of what GeoArrow in Java should do: things like JTS and the Arrow Java bindings should almost certainly be connected; however, I imagine that those connections should be scoped such that somebody can use this to do something useful without either.

I'm happy to help review any of this or help port parts of other bindings into Java.

cc @jiayuasu

[jiayuasu]

I agree with your suggestion @paleolimbot on adding JTS and Arrow-Java as dependency.

And the implementation should somehow be designed in a way that Sedona can easily re-use it (with slight modification) to generate the native encoding in GeoParquet.

Any opinions? @Kontinuation @zhangfengcdt

[Kontinuation]

We need (1) Arrow schema definition of various geometry types and (2) conversions between arrow and JTS geometry from the minimal implementation. Parquet reading/writing could be handled by Sedona and does not need to be part of the minimal implementation.

[paleolimbot]

Arrow schema definition of various geometry types

Naturally! I assume the output here would be an ArrowType or Field but there would also be org.geoarrow.core.DataType that implements the GeoArrow-specific bits.

conversions between arrow and JTS geometry from the minimal implementation.

Got it! I off the top of my head, this would probably go from arrow ValueVector -> org.geoarrow.core.ArrayData (simpler...a geoarrow type plus the relevant buffers) -> JTS. My intuition would be to implement something like a ArrowJTSVisitor interface that would let the implementation call a Java method for each JTS Geometry in the output (the other option would be to build a Vector or array of JTS Geometry).

should somehow be designed in a way that Sedona can easily re-use it (with slight modification) to generate the native encoding in GeoParquet.

I don't know if you are using Arrow's scanner or not (or heavily relying on JTS geometries internally), but the org.geoarrow.core.ArrayData should let you create something that can be converted to JTS Geometries in the design I had in mind.

I think there will probably another issue with CRS handling. Either we need to implement a way to create PROJJSON in Java or allow some other CRS representation in the GeoArrow specification (or maybe you have this part already from working with GeoParquet). I think consuming PROJJSON is quite a bit simpler but would have to be implemented either way.

[msbarry]

Hello! Would you need to implement a projjson parser as well? Or does one exist for java somewhere else? I'm trying to add geoparquet support to planetiler and am having a hard time finding one.

[paleolimbot]

Technically all we need here is an interface that a CRS-like object can implement that gets us PROJJSON; however, I think this repo would be a good home for PROJJSON <-> WKT2 (although I gather that even WKT2 doesn't have excellent support in Java, although that may have changed).

[willcohen]

Hi all -- native PROJ is (somewhat) compiled and wrapped for the JVM via https://github.com/willcohen/clj-proj/tree/main. It requires firing up FFI of the wrapped C library, but once that overhead is handled, it can/could consume PROJJSON using the original parser, though I probably need to wrap a few more functions to achieve what you all are looking for. Are you actually trying to perform the transformations themselves, or just to associate projection representations with a JVM CRS object?

In parallel I'm also working through trying to get JTS objects and projections and serialized objects sorted out into something a little more systematic (locationtech/jts#1101), so my helping to get this stubbed out project into something a little more solid is very much in my interest, and I'd be happy to help!