Publication System - Scale Testing v1

[lliming]

Vision: Publication system handles realistic CMIP6 datasets (real entries, realistic numbers of files, realistic number of replicas)

Acceptance criteria: Realistic datasets can be published end-to-end with robust error reporting and logging.

• Using all of the POC components, deploy all in AWS and run tests using CMIP6 datasets from existing ESGF indices.
• Create issues as they're encountered

NOTE: This issue is not looking at many other kinds of scaling! (E.g., total number of datasets in catalog, number of enrolled teams, individual users, concurrent actions, volume of actions, etc.)