Skip to content

Add docs page on how to use S3 data.#6951

Merged
ukmo-ccbunney merged 12 commits intoSciTools:mainfrom
pp-mo:s3_advice
Feb 24, 2026
Merged

Add docs page on how to use S3 data.#6951
ukmo-ccbunney merged 12 commits intoSciTools:mainfrom
pp-mo:s3_advice

Conversation

@pp-mo
Copy link
Member

@pp-mo pp-mo commented Feb 18, 2026

Closes #6374

@pp-mo pp-mo mentioned this pull request Feb 18, 2026
4 tasks
@codecov
Copy link

codecov bot commented Feb 18, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.20%. Comparing base (5d91446) to head (ecbca4e).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #6951   +/-   ##
=======================================
  Coverage   90.20%   90.20%           
=======================================
  Files          91       91           
  Lines       24886    24886           
  Branches     4672     4672           
=======================================
  Hits        22449    22449           
  Misses       1667     1667           
  Partials      770      770           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@pp-mo pp-mo marked this pull request as ready for review February 18, 2026 16:08
Copy link
Contributor

@ukmo-ccbunney ukmo-ccbunney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey @pp-mo
Generally this reads really well.
I have some small suggested changes.

The one issue I had though was that I could not install the s3-fuse package as detailed. I can see s3fs packages on pip and conda, but I think that these are not the fuse versions. This means I have not actually tested the instructions at this point.

There is a lot of ambiguity around the s3 packages it seems! If you can point me to the correct conda/pip package then I will test.

Copy link
Member Author

@pp-mo pp-mo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry @ukmo-ccbunney I had a few suggestions of my own for your consideration,
which I apparently failed to post till now

pp-mo and others added 4 commits February 20, 2026 15:30
Co-authored-by: Chris Bunney <48915820+ukmo-ccbunney@users.noreply.github.com>
Co-authored-by: Chris Bunney <48915820+ukmo-ccbunney@users.noreply.github.com>
Co-authored-by: Chris Bunney <48915820+ukmo-ccbunney@users.noreply.github.com>
Co-authored-by: Chris Bunney <48915820+ukmo-ccbunney@users.noreply.github.com>
Copy link
Contributor

@ukmo-ccbunney ukmo-ccbunney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

OK - so the instructions work for me once I installed s3fs-fuse via conda.
However, see comments regarding pip/PyPI - you can't install it this way (it is not a python package).

@pp-mo
Copy link
Member Author

pp-mo commented Feb 21, 2026

Thanks @ukmo-ccbunney for comprehensive + diligent review !

I think I've now covered all the points raised so far,

except what I commented about not giving any details of the credentials management : My thought there was that anyone using this would presumably have already dealt with that :
Happy to reconsider, if you think worhwhile.

Based on various earlier comments, I have tried to rationalise the use of bold "**" for names and code-format "``" for commands etc. Hope this now works for you but, again, open to discuss

Copy link
Contributor

@ukmo-ccbunney ukmo-ccbunney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Just a few things, then I think we're good!

pp-mo and others added 2 commits February 24, 2026 14:48
Co-authored-by: Chris Bunney <48915820+ukmo-ccbunney@users.noreply.github.com>
@pp-mo
Copy link
Member Author

pp-mo commented Feb 24, 2026

Thanks @ukmo-ccbunney .
Hopefully that fixes it!

Copy link
Contributor

@ukmo-ccbunney ukmo-ccbunney left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great! Looking good. LGTM!

@ukmo-ccbunney ukmo-ccbunney enabled auto-merge (squash) February 24, 2026 19:11
@ukmo-ccbunney ukmo-ccbunney merged commit d79be4e into SciTools:main Feb 24, 2026
21 checks passed
@pp-mo
Copy link
Member Author

pp-mo commented Feb 25, 2026

Great! Looking good. LGTM!

Thanks @ukmo-ccbunney !
And a great relief to get this done before #6868. Let them sort it ! 👿

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

Status: Done

Development

Successfully merging this pull request may close these issues.

Extend Iris loading to accept S3 URLs

2 participants