implementation for archive extension

[kmodali]

Handling STAC items with tar assets, either single tar or list of tars.

[jsignell]

Thank you for opening a PR @kmodali! It might take me a few days to get around to reviewing, but it is on my list!

[jsignell]

I recognize that this PR might be in a partial state, but now that I've (finally!) read it over I feel pretty convinced that this logic doesn't belong in xpystac. It should be the responsibility of the zarr backend to make sense of an archived zarr file. So while xpystac might need to add a little logic to understand how to hand off metadata from the archive extension fields it shouldn't be in charge of any caching or extracting.

[jsignell]

This should always evaluate to true since this function is registered for the case where obj is an Asset.

[kmodali]

@jsignell Thank you very much for reviewing this PR and providing valuable suggestions. Subsequently the corresponding changes for 'archiveextension' have been made. Also the scope of this PR now extends across zarr-python , xpystac and pystac.

zarr-python

• https://github.com/kmodali/zarr-python/tree/rel315 has the implementation for 'tarstore' and some simple tests are provided under https://github.com/kmodali/zarr-python/tree/rel315/tests/test_store/test_tar.
• The creation of multiple groups and consolidating their metadata is not working yet. However, the reading has no issues.

pystac

• https://github.com/kmodali/pystac/tree/archiveextension has 'archveextension' implementation.
• This can be tested in tandem with the implementation in 'xpystac'. ( @gadomski ).

xpystac

• https://github.com/kmodali/xpystac/tree/archiveextension has the 'archiveextension' implementation.
• The tests for demonstrating the usage of the complete chain -- STAC --> pystac --> xpystac --> xarray --> zarr-python are provided under https://github.com/kmodali/xpystac/tree/archiveextension/tests , in the form of two notebooks. As mentioned above the 'tarstore' has issue with creation of multiple groups, therefore the 'zarr stores' and the corresponding 'tar stores' are created using 'createZarrAndTar_Stores.ipynb'.
• The accessibility of assets with 'archiveextension' is demonstrated in 'multiTarStoreRead_From_xpystac.ipynb'.

[kmodali]

zarr-python ( update )

• https://github.com/kmodali/zarr-python/tree/rel315 has been updated to handle the creation of multiple groups in a TarStore and consolidating their metadata in the store level 'zarr.json'.

[jsignell]

Thanks for writing this up @kmodali. When adding new code to this repo it should have tests that look like those in https://github.com/stac-utils/xpystac/tree/main/tests

[kmodali]

@jsignell Added 'tests/test_archiveextension.py' as suggested. Thank you.