Skip to content

Enable Temporary Credentials for AWS S3 Access #3136

@alexandervladsemenov

Description

@alexandervladsemenov

When the HDF5 ROS3 driver is invoked, only the region, access key ID, and secret key are currently passed. However, to access NASA Earthdata S3 buckets, a session token is also required. While the HDF5 ROS3 driver does support a session token, it is not currently being utilized.

It would be beneficial to:

  • Add support for passing a session token to the ROS3 driver.

  • Parse all potential sources of credentials (e.g., environment variables and AWS config files) to determine the most appropriate set of credentials for accessing multiple buckets.

  • Add support for S3 URIs such as:
    s3://ob-cumulus-prod-public/PACE_OCI.20240411T000331.L1B.V3.nc

I’ve created a development branch to address this issue:
👉 https://github.com/alexandervladsemenov/netcdf-c/tree/temp_credentials

The code has been tested and is functional. However, I believe the current mechanism for parsing URLs/URIs in netCDF could benefit from a broader redesign. I'd like to get input from the netCDF development team before proceeding further.

I’ll submit a draft pull request to facilitate feedback and discussion.

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions