Application to include the CAM6+DART Reanalysis in AGU's '100 Most Impactful Datasets' special issue

[kdraeder]

The application is due 2025-10-10, so this is a long head start.
Jeff suggested that (someone on) the team should submit the nomination, but multiple nominations for the same dataset are welcome. This Discussion can provide the description of the dataset, and nominators would provide their personal reasons for nominating.

I've included a condensed version of the list of information requested in the application, with my first draft responses. I did not provide anything for "how I interact with the dataset" or "examples of how it has been reused" (why not "used"?).

---

Describe how the dataset is used to study and understand the impact of Earth, space, and environmental science on dimensions of people, planet, and prosperity.
This includes research, policy making, analysis, activities around environmental disasters, and conservation - or other related. See the 2030 Agenda for Sustainable Development and Geosciences in Action

People - Positive impact on Individuals: education, training, disaster response and prediction

Planet - Positive impact on physical systems: geophysical phenomena, conservation, climate, environmental indicators

Prosperity - Positive impact on socioeconomic systems: economic good, social equity, humanitarian relief, community resilience

You may nominate any dataset you use for your research, analysis, or decision making. This can be a dataset you and your team created,

---

Information requested in the form

• Nominator(s) Information: Name, email, ORCID, Affiliation/Organization
• Dataset information:
  • Name:
    • CAM6 Data Assimilation Research Testbed (DART) Reanalysis
  • Authors/creators:
    • Kevin Raeder (1)
    • Timothy J. Hoar (4)
    • Mohamad El Gharamti (1)
    • Benjamin K. Johnson (4)
    • Nancy Collins (4)
    • Jeffrey L. Anderson (1)
    • Jeff Steward (2)
    • Mick Coady (4)
    • (1) National Center for Atmospheric Research, CISL/DAReS, Boulder, CO, 80305, USA
    • (2) Spire Global Inc., 1690 38th St, Boulder, CO, 80301, USA
    • (3) National Center for Atmospheric Research, CISL/CSG, Boulder, CO, 80305, USA
    • (4) Retired
  • Link/Location (Persistent Identifier or URL):
    • https://rda.ucar.edu/datasets/d345000/
    • DOI: 10.5065/JG1E-8525
  • Repository Name where dataset is hosted:
    • NSF NCAR's Research Data Archive
  • Repository Persistent Identifier:
  • Repository data managers / curators / group-name that support the dataset (optional):
    • Chi-Fan Shih, Bob Dattore, Dave Stepaniak, Riley Conroy, Doug Schuster,
    • Thomas Cram, Zaihua Ji, Harsha Hampapura, Calie Payne
• Short description of the dataset being nominated.
  • Target about 1500 characters.

This ensemble Kalman filter reanalysis used a CAM6 configuration of the Community Earth System Model (CESM), several million observations per day, and the Data Assimilation Research Testbed (DART) to generate gridded descriptions of the lower atmosphere which have more uniform coverage than the observations and smaller errors than a free-running model. The data saved from this global ~1-degree-resolution 80-member ensemble span 2011-2020. They include ensembles of:
a) Sub-daily, real world, atmospheric forcing for use by all of the nonatmospheric models of CESM.
These files can be read by CESM which will distribute the data to the components.
b) Weekly CAM6+CLM5+CICE+MOSART (no ocean) restart file sets
for starting CESM hindcast experiments.
c) 6-hourly hindcast estimates of the assimilated observations,
which could be used as a labeled training dataset for machine learning.
d) 6-hourly ensemble-mean gridded atmospheric analyses.
e) 6-hourly land-model plant-growth variables.
This data can be used for hindcast studies and data assimilation using component models of CESM; CAM6, CLM5, CICE5, POP2. MOM6, MOSART, and CISM; and non-CESM Earth system models. There is also a dataset which shows the quality and uncertainty of the other datasets as a function of location, time, and atmospheric variable. This large dataset (~120 Tb) has a unique combination of a large ensemble, high frequency, and multiyear time span, which provides opportunities for robust statistical analysis and use as a machine learning training dataset.

• Justification for inclusion in AGU commentary
  • Why is the nominated dataset impactful in your own words or in your opinion, in no more than 3000 characters. If accepted, this text will be included with the dataset information.
  • One aspect to consider is what couldn't we do, know, or study if the dataset wasn't available? Please use the dimensions of impact as a guide.

Data assimilation is the tool we use to understand everything understandable. It combines noisy scattered observations with coherent but incomplete theories to give us the best available understanding. Ensemble data assimilation is especialy valuable when applied to highly complex systems such as the Earth's biosphere, because it scales beyond an individual's capability to assimilate information and provides estimates of the uncertainty (confidence) in the understanding.

The most widely-used subset is the ensemble atmospheric forcing, which provides the time-varying upper boundary conditions (BC) for "surface" models, such as the land (Community Land Model) and ocean (Parallel Ocean Program and Modular Ocean Model). This forcing represents the actual weather during 2011-2020 and the ensemble presents realistic variance (uncertainty), which the models must be able to handle. The ensemble of forcing also maintains spread in the surface model ensemble, which is essential for ensemble data assimilation.
The 80-member ensemble of restart files provide initial conditions (ICs) for the surface model ensembles (excluding the ocean).

This combination of ICs and BCs enables ensemble data assimilation experiments without needing to run an expensive ensemble of atmospheric models for each one. This reduces the computational, human, time, and energy resources required for research which benefits humanity in multiple ways:
a) improving models for forecasting common and extreme events, and climate change,
in the atmosphere, ocean, land, ecosystems, and cryosphere.
b) improving understanding of the physics, chemistry and ecology of the Earth system,
c) making efficient use of the extensive observations for which we are paying,
d) identifying which additional observations would be helpful,
e) understanding the uncertainties in the data and forecasts.
They can be used by models ranging from spatially global to single column.

The subset consisting of the actual observations used in the assimilations and the estimated observations generated along the way are an underutilized resource. There are ~10 billion actual observations, each one labeled with a wealth of metadata; whether or not it was assimilated and why, observation type (e.g. wind) and source (e.g. aircraft), location, and time. Labeled datasets of this size of an Earth system are rare.

These datasets are freely available and are supported by the DAReS group at NSF NCAR.

• Short description of how you interact with this dataset.
  • No more than 1500 characters. This is intended to be your personal experience.
• Reference publication that describes the dataset. Optional

Raeder, K., Hoar, T.J., El Gharamti, M. et al.
A new CAM6+DART reanalysis with surface forcing from CAM6 to other CESM models.
Sci Rep 11, 16384 (2021). https://doi.org/10.1038/s41598-021-92927-0

• Examples of how this dataset has been reused,
  • any bibliographies listing instances of reuse,
  • or any references to derived data products/datasets. Optional

[braczka]

CLM based CAM reanalysis applications. This is primarily a placeholder for now, but I can add more information later:
1a) Project: NASA-CMS grant: NCAR/University of Utah/JPL, Xueli Huo, John Lin
1b) Purpose: Using CAM6 with CLM-DART for carbon monitoring across Western US
1c) Products: Huo et al., (in prep)

2a) Project: Cornell led NASA-CMS project. Ying Sun, Zhenqi Luo, Jiameng Lai
2b) Purpose: Using CAM6 with CLM-DART for carbon monitoring, forest restoration across East and Sub-Saharan Africa
2c) Products: Lai et al., (in prep)

3a) Project: NASA-ABOVE, David Moore, Andy Fox, Xueli Huo
3b) Purpose: Using CAM6 with CLM-DART for characterizing vulnerable ecosystems (Arctic-Boreal Canada)
3c) Products: doi:10.1029/2024JG008004

4a) Project: NCAR ESPAT. Steve Yeager, Sasha Glanville
4b) Purpose: Using CAM6 as initialization for improved sub-seasonal atmospheric forecasting.
4c) Products: ?? Several presentations

5a) Project: India Institute of Technology, Devavat Chiru Naik
5b) Purpose: CAM6 applied to CLM-DART for India drought monitoring system
5c) Products: EGU 2025 presentation, manuscript in prep

6a) Project: Daniel Hagan, Ghent University
6b) Purpose: CAM4 applied to CLM-DART for China soil moisture analysis
6c) Products: Presentations, manuscript in prep.

7a) Project: University of Arizona/ NCAR: Andy Fox, Dave Moore
7b) Purpose: Using CAM6 with CLM-DART for global carbon uptake estimates.
7c) Products: doi:10.1029/2022JG006830

8a) Project: NASA-CMS, University of Utah, Brett Raczka
8b) Purpose: Using CAM4 with CLM-DART for improved carbon cycling across Western US.
8c) Products: doi:10.1029/2020MS002421
((https://doi.org/10.3334/ORNLDAAC/1856))

[kdraeder]

@braczka Thanks for the good start on this!