Accompanying slides can be found here.
This project demonstrates:
- UnionAI/Flyte to orchestrate e2e workflows to ingest and build xarrays
- GDAL's GTI with rioxarray to efficiently build large Xarray datasets
- Flyte-Dask plugin for dask clusters at your fingertips
Not sure where to start? Look to the flyte/build.py:build_dataset_mosaic_workflow
workflow for the main entrypoint defining a workflow to build mosaics.
- add basic github actions for testing, linting, pre-commit, building and pushing container to gchr?
- gh is installed + gh auth login has read:packages permissions
- mamba, conda-lock, docker
- docker desktop or colima for
make build-push - ensure visibility of ghcr image is public.
- union auth create --host ... if using a unionai deployment