GCAFS prep emissions update - fire emissions #3713
Conversation
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
There was a problem hiding this comment.
Pull Request Overview
This pull request updates aerosol configuration and emissions handling for the UFS-Aerosols system by consolidating file management and enhancing modularity. Key changes include:
- Removing legacy logic for generating the AERO_ExtData.rc file and consolidating emission configuration files.
- Adding a link to chemical input files in forecast_predet.sh and updating configuration templates in both aerosol and chemical configuration files.
- Refactoring configuration files to improve clarity and maintainability.
Reviewed Changes
Copilot reviewed 10 out of 10 changed files in this pull request and generated 1 comment.
Show a summary per file
| File | Description |
|---|---|
| ush/parsing_namelists_GOCART.sh | Removed legacy dynamic generation of AERO_ExtData.rc. |
| ush/forecast_predet.sh | Added linking for CHEM_INPUT to ensure chemical input file availability. |
| parm/ufs/gocart/ExtData.* | Removed obsolete emission configuration files, consolidating them into AERO_ExtData.rc. |
| parm/ufs/gocart/AERO_ExtData.rc | Updated emission entries to use chemical input paths and adjusted emission factors. |
| parm/chem/chem_emission.yaml.j2 | Updated templating loops for QFED variables and file copying operations. |
| dev/parm/config/gfs/config.com | Added new chemical template paths. |
| dev/parm/config/gcafs/config.aero.j2 | Reorganized aerosol sections and introduced new aerosol emission settings. |
bbakernoaa
force-pushed
the
feature/prep_emis_rework
branch
from
2e06fb3 to
b534a8b
Compare
|
@bbakernoaa is this ready now after conflicts are resolved? |
|
@CoryMartin-NOAA Yes all tests passed on wcoss2. I submitted with this command ./generate_workflows.sh -A GEFS-DEV -E -G -C -S /lfs/h2/emc/ptmp/barry.baker/GW_RTs |
emcbot
added
CI-Gaeac6-Ready
aerorahul
left a comment
aerorahul
left a comment
There was a problem hiding this comment.
A few comments that should not be breaking anything, I hope
| export tasks_per_node=$(( max_tasks_per_node / threads_per_task )) | ||
| export threads_per_task=${max_tasks_per_node} | ||
| export tasks_per_node=1 | ||
| export OMP_NUM_THREADS="${threads_per_task}" |
There was a problem hiding this comment.
This is usually reserved till the executable is called. Perhaps in a follow-up PR.
| export memory="1GB" | ||
| export threads_per_task=${max_tasks_per_node} | ||
| export tasks_per_node=1 | ||
| export OMP_NUM_THREADS="${threads_per_task}" |
| export memory="1GB" | ||
| export threads_per_task=${max_tasks_per_node} | ||
| export tasks_per_node=1 | ||
| export OMP_NUM_THREADS="${threads_per_task}" |
| # Create the ChemInput directory in the local run directory | ||
|
|
||
| # if [[ ! -d "${DATA}/ChemInput" ]]; then mkdir -p "${DATA}/ChemInput"; fi | ||
|
|
||
| # # Copy Fire Emission Files ChemInput directory | ||
| # local current | ||
| # local YYYYMMDDHH | ||
| # current="${current_cycle_begin}" | ||
| # cycleend=$(date -d "${current_cycle_end:0:8} ${current_cycle_end:8:2} +24 hour" +%Y%m%d%H) | ||
| # while [[ "${current}" -le "${cycleend}" ]]; do | ||
| # # Validate current is a valid date string | ||
| # if ! YYYYMMDD=$(date -d "${current:0:8} ${current:8:2}:00:00" +%Y%m%d 2>/dev/null); then | ||
| # echo "FATAL ERROR: Invalid date string '${current}' in GOCART_predet, ABORT!" | ||
| # exit 1 | ||
| # fi | ||
| # local FireEmisFile="${COMIN_CHEM_INPUT}/FIRE_EMIS_${YYYYMMDD}.nc" | ||
| # if [[ -f "${FireEmisFile}" ]]; then | ||
| # cpreq "${FireEmisFile}" "${DATA}/ChemInput/" | ||
| # else | ||
| # echo "FATAL ERROR: GOCART input file '${FireEmisFile}' does not exist, ABORT!" | ||
| # exit 1 | ||
| # fi | ||
|
|
||
| # # Increment by 1 day | ||
| # current=$(date -d "${current:0:8} ${current:8:2} +1 day" +%Y%m%d%H) | ||
| # done | ||
|
|
||
| # # Copy NXS Emission Files ChemInput directory | ||
| # # NXS files are hourly, so we need to loop through each hour in the cycle | ||
| # current=$(date -d "${current_cycle_begin:0:8} ${current_cycle_begin:8:2}" +%Y%m%d%H) | ||
| # cycleend=$(date -d "${current_cycle_end:0:8} ${current_cycle_end:8:2} +1 hour" +%Y%m%d%H) | ||
| # while [[ "${current}" -le "${cycleend}" ]]; do | ||
| # if ! YYYYMMDD=$(date -d "${current:0:8} ${current:8:2}:00:00" +%Y%m%d 2>/dev/null); then | ||
| # echo "FATAL ERROR: Invalid date string '${current}' in GOCART_predet, ABORT!" | ||
| # exit 1 | ||
| # fi | ||
| # local NXSFile="${COMIN_CHEM_INPUT}/${NEXUS_DIAG_PREFIX}.${YYYYMMDD}.nc" | ||
| # if [[ -f "${NXSFile}" ]]; then | ||
| # cpreq "${NXSFile}" "${DATA}/ChemInput/" | ||
| # else | ||
| # echo "FATAL ERROR: GOCART input file '${NXSFile}' does not exist, ABORT!" | ||
| # exit 1 | ||
| # fi | ||
| # # Increment by 1 hour | ||
| # current=$(date -d "${current:0:8} ${current:8:2} +1 hour" +%Y%m%d%H) | ||
| # done | ||
|
|
There was a problem hiding this comment.
why are we keeping this large commented section?
There was a problem hiding this comment.
@bbakernoaa Can you please add yourself as a CODEOWNER for this and the next file please?
| logger.error(f"Error combining QFED files: {e}") | ||
| logger.error(f"Traceback: {traceback.format_exc()}") |
There was a problem hiding this comment.
Should this be a Warning? An Error should stop processing, a warning can continue with processing
| logger.info(f"Number of forecast hours: {nforecast_hours}") | ||
|
|
||
| # Create start date based on SDATE | ||
| self.start_date = self.task_config["CDATE"] - to_timedelta('24H') # include previous day |
There was a problem hiding this comment.
I think we eliminated the use of CDATE. Here, you can use current_cycle
| logger.info(f"Start date: {self.start_date}") | ||
|
|
||
| # end date = SDATE + nforecast hours + 36 | ||
| self.end_date = self.task_config["CDATE"] + to_timedelta(f'{nforecast_hours + 36}H') |
| nforecast_hours = self.task_config["FHMAX_GFS"] | ||
|
|
||
| # Create start date based on SDATE | ||
| self.start_date = self.task_config["CDATE"] |
| # Create start date based on SDATE | ||
| self.start_date = self.task_config["CDATE"] | ||
| self.total_hrs = nforecast_hours + 3 | ||
| self.end_date = self.task_config["CDATE"] + to_timedelta(f'{self.total_hrs}H') |
There was a problem hiding this comment.
current_cycle in place of CDATE.
|
@aerorahul I don't think any of these will break it. I'll fix it first thing tomorrow |
emcbot
added
CI-Hercules-Running
|
C96_gcafs_cycled FAILED on Hercules (pipeline ID: 6185) In directory: This failure was detected automatically by global-workflow's CI/CD Pipeline |
emcbot
added
CI-Hercules-Failed
|
C96_gcafs_cycled_noDA FAILED on Gaeac6 (pipeline ID: 6185) In directory: This failure was detected automatically by global-workflow's CI/CD Pipeline |
emcbot
added
CI-Gaeac6-Failed
|
C96_gcafs_cycled FAILED on Gaeac6 (pipeline ID: 6185) In directory: This failure was detected automatically by global-workflow's CI/CD Pipeline |
|
@aerorahul I fixed why it was crashing. The cycledef in the prep_emissions job was changed to |
8 participants
Description
Resolves #3712
Resolves #3827
Resolves #4248
This pull request introduces significant updates to the aerosol configuration and emission handling in the UFS-Aerosols system, streamlining file management, improving modularity, and consolidating configuration files. Key changes include restructuring aerosol configuration files, enhancing biomass burning emissions handling, and updating the chemical input file linking process.
Aerosol Configuration Updates:
config.aero.j2to organize settings into distinct sections for aerosol and emission configurations, including the addition of new variables likeAERO_EMIS_FIRE_VERSIONandAERO_EMIS_FIRE_HIST. [1] [2]Emission Data Management:
ExtData.gbbepx,ExtData.qfed, andExtData.none) into a single file,AERO_ExtData.rc, for better modularity and maintainability. Removed redundant and unused emission entries. [1] [2] [3] [4]Chemical Input File Handling:
CHEM_INPUT) inforecast_predet.shto ensure proper file availability during runtime.Removal of Legacy Code:
AERO_ExtData.rcinparsing_namelists_GOCART.sh, as the new consolidated configuration file eliminates the need for dynamic generation.Additional Updates:
config.comto include new template paths for chemical restart and input files, improving file organization.Type of change
Change characteristics
How has this been tested?
Checklist