Hosting experiment repo on Github. Workflow for users for publications.

[chrisb13]

This recent support request from @georgyfalster highlights that we should have clear access hive instructions for the workflow for how users and community contributors could host their experiments on GitHub.

Quite a few instructions have already been written for various use cases/users:

1. https://forum.access-hive.org.au/t/utilising-github-effectively-for-experiments-and-configurations/2027
2. https://github.com/ACCESS-Community-Hub/access-esm1.6-dev-experiments
3. https://forum.access-hive.org.au/t/running-model-experiments-with-payu-and-git/2285

I just think these would be best consolidated onto the access hive, as it's something we want to be encouraging all our users to do. It would be good to tie it into how you then can link it with Zenodo for publication purposes. Note that 1/2 differ above in using lots of single repos vs a single repo with lots of branches (for each experiment). So the pro/cons of that choice are worth making clear.

ping @aidanheerdegen @ccarouge @blimlim who have been involved in the above. And @kdruken @paigem @atteggiani for interest?

[ccarouge]

I agree this is a topic worthy of better documentation. Since the workflow is tied to payu's features, should this be somehow part of payu's documentation? I don't mean to say it shouldn't be in the Hive docs, what I mean is it should be clearly shown to be a payu workflow.

[atteggiani]

I agree this might be better suited for payu documentation rather than the Hive Docs.
I think what we could do in the Hive Docs, is have those links (or any documentation if a better one is written) included within the payu tutorials.
By the way, I can see one of the links (running-model-experiments-with-payu-and-git) is already included in the payu tutorials.

[chrisb13]

Thanks for your input @atteggiani.

On your comment

better suited for payu documentation rather than the Hive Docs.

I don't think @ccarouge was saying that as such

I don't mean to say it shouldn't be in the Hive docs, what I mean is...

In any case, I think we're in the right direction here. This kind of documentation would need close collaboration with the Payu / release team but I think in practice most users don't find those doc's and this workflow is of high importance if we want our users to share what they are doing. In my view, we want to make it routine for users to be sharing model configurations (and analysis code etc) when they publish. Making it easy for them will really help demonstrate our organisations impact / improve community collaboration / build better tools etc. Happy to discuss.

[atteggiani]

Thank you for the clarification @chrisb13.

I agree it's an important piece of documentation for users to have and it should be easily findable.
I still think they are best suited in the payu tutorials page, as they are specifically related to payu, and they are tutorials on how to perform specific tasks with it or they explain suggested workflows.

A way to try and make them more findable, is placing links to the payu tutorials in the "Run model" pages that refer to model to be run with payu.

Do you have any other suggestions?

[chrisb13]

I think the problem at the moment is that it's quite fragmented. I'm actually not sure what you mean by

the payu tutorials page
?

Can you be specific?

I've found these pages:

1. https://forum.access-hive.org.au/t/access-om2-payu-tutorial/1750
2. https://payu.readthedocs.io/en/latest/
3. https://github.com/payu-org/payucourse?tab=readme-ov-file
4. https://forum.access-hive.org.au/t/payu-a-workflow-manager-for-some-access-models/1098

I think you might mean, from this earlier post:

I think what we could do in the Hive Docs, is have those links (or any documentation if a better one is written) included within the payu tutorials.

Which are 1/4 above. Whilst great, they don't speak to the issue I'm raising. Do any links currently exist to the payu official docs?

Sidepoint: another point of confusion is the payu-org/payu and then our fork of it.

I still think they are best suited in the payu tutorials page, as they are specifically related to payu

I think that's only partially true as it would use zenodo / github as well.

[atteggiani]

Yes I meant the Hive Docs page that I linked in my comment above.

Which are 1/4 above.

If you think more tutorials should be added, we can add them to that page.

Do any links currently exist to the payu official docs?

In all the Prerequisites of the models, in the Payu section, there is:

Payu is a workflow management tool for running numerical models in supercomputing environments, for which there is extensive documentation.

that links to payu documentation.
There's also a link in the References of those "How to run" pages