WRI-CAIT - S3 - Mapping

[S-AI-F]

Once mapped json file stored in github repository is validated, you can update your mapping script by replacing input/output data with s3 buckets.

You have a specific folder by data source where you can put raw data and mapped generated data (in staging folder).

Doc:

• data staging model: https://github.com/OpenGeoScales/ogs-connectors/wiki/2---Data-Model---Staging
• data staging storage: https://github.com/OpenGeoScales/ogs-connectors/wiki/Datalake-AWS-S3

[CJeannesson]

This has been done, but without relying on a Python script.

[S-AI-F]

Hi @CJeannesson
Can you change please the mapping script by reading raw data from S3 and writing mapped data in S3?
Since, you are the first to to make this change, can you write a small tutorial of connecting python to S3 to share with the team?
Here is the dedicated wiki page: https://github.com/OpenGeoScales/ogs-connectors/wiki/Datalake-AWS-S3