Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

cam-se fails if CLDICE, CLDLIQ, or Q (or PS) is not in the state vector #662

Open
kdraeder opened this issue Apr 5, 2024 · 10 comments · May be fixed by #776
Open

cam-se fails if CLDICE, CLDLIQ, or Q (or PS) is not in the state vector #662

kdraeder opened this issue Apr 5, 2024 · 10 comments · May be fixed by #776
Assignees
@kdraeder @hkershaw-brown
Labels
Bug Something isn't working CAM-SE Community Atmosphere Model Spectral Element Core pmp possible Marlee project

Comments

@kdraeder
Copy link
Contributor

kdraeder commented Apr 5, 2024

Describe the bug

  1. List the steps someone needs to take to reproduce the bug.
    Build a cam-se filter.
    Set model_nml:state_variables to a set of variables that is "missing" one or more of the moisture variables.
    Assimilate any appropriate observation (Feature request: CAM SE Manhattan #343 may provide files that are helpful for setting up an assimilation).
  2. What was the expected outcome?
    The existence of state_variables in the namelist implies that the user is free to define the model state,
    so the expected outcome is that the observations will be assimilated.
  3. What actually happened?
    All observations fail in the forward operator (QC=4).

Error Message

There is not enough diagnostic messaging from model_mod to learn the cause of the failure.
Adding messages about the various istatus contents leads to the discovery that build_dry_mass_pressure_columns requests values for QTY_SPECIFIC_HUMIDITY, QTY_CLOUD_LIQUID_WATER,
and QTY_CLOUD_ICE. If one of those fails, the failure code is immediately returned to the calling routine
without any diagnostic message. The calling routine converts the code to a generic failure code,
which eventually is converted into QC=4, which is not very informative, given the large number of ways
that the forward operator can fail.

Which model(s) are you working with?

cam-se
A similar issue applies to cam-fv too, because PS is required (in both).
No one has ever excluded PS and reported a problem.

Version of DART

Which version of DART are you using?
Any version after the refactoring which uses cam-common-code and after build_dry_mass_pressure_columns
was added. (I don't see it mentioned in the git log output, so I don't know when it was installed.)
The version I modified and sent to Nuo Chen for testing is v9.12.refactored-1915-gb3e00330f.
If a cleaner version number is needed, I'll work on finding one.

Have you modified the DART code?

I added diagnostic prints, but did not change any calculations or procedure calls.
There are several possible solutions to this problem:

  1. Tell users in documentation that state_variables must include those 3 moisture variables. This is minimal effort, but unreliable.
  2. Add code to check for the presence of the moisture variables and fail if one is missing.
  3. Add printed messages (controlled by debug_level) that list useful debugging information: subroutine, istatus, related variables,... This can save hours to days of debugging. If this is part of the chosen solution, I can provide a version that does some of this.
  4. Define (hardcode) a minimal set of state variables which are always in the state and allow users to add variables through the namelist.

A question related to 1. is whether those 3 variables are all that are required to correctly calculate the dry mass pressures.

Build information

Please describe:

  1. The machine you are running on: any machine
  2. The compiler you are using: any compiler
@kdraeder kdraeder added Bug Something isn't working CAM-SE Community Atmosphere Model Spectral Element Core labels Apr 5, 2024
@hkershaw-brown
Copy link
Member

Hey Kevin, v9.12.refactored-1915-gb3e00330f means 1915 commits after version v9.12 so we should chat and get you working on the main branch again.

@hkershaw-brown
Copy link
Member

the dry mass pressure came in here: 1982c75
There are some notes about the 'precise' setting here:
#343 (comment)

@hkershaw-brown
Copy link
Member

Thanks for reporting this, Kevin (and Nuo). Before doing a pull request to fix for this, I'd like to get a spec for what the dry_mass_vertical_coordinate options are, what namelist options are needed and what state qtys are needed for the namelist options. Ideally we would have done this when the original code was written, but the second best time to do this is now.

@kdraeder
Copy link
Contributor Author

kdraeder commented Apr 5, 2024

@hkershaw-brown thanks for digging up the SHA.
I didn't dig into the dry mass code enough to be able to write a spec yet, and I probably won't be able to work on this until April 15.
Someone else might be able to do it sooner or faster.
Which solution looks best to you?

@hkershaw-brown
Copy link
Member

Spec first.

@kdraeder
Copy link
Contributor Author

I've written a spec and put it in a G-doc in DAReS team > Spec > Handling calculation of total pressure in cam-se

There may be some confusion, and some connection, between this issue and
one that @jlaucar says has been lurking for several years.
This issue (#662) seems straightforward to me; if a user leaves out any of the moisture fields
from the state definition, then all of the obs will get QC=4.
Jeff's related issue may be mentioned in the cam-se/model_mod:

! Note that other tracers in the dry mass cam/se have a dry mixing ratio, not specific mixing ratio (although
! Peter plans to change this for consistency).
!SENote: The following should all be 0, but A's at levels 2, 3, 4, and 5 are not.
! This is confirmed in the caminput.nc file and is a PROBLEM.
! Some tests on the A and B coefficients
!do k = 1, nlevels
   !write(*, *) k, (grid_data%hyai%vals(k) + grid_data%hyai%vals(k+1))/2.0_r8 - grid_data%hyam%vals(k)
   !write(*, *) k, (grid_data%hybi%vals(k) + grid_data%hybi%vals(k+1))/2.0_r8 - grid_data%hybm%vals(k)
!enddo

@kdraeder kdraeder mentioned this issue Apr 18, 2024
Open
@hkershaw-brown hkershaw-brown added the pmp possible Marlee project label Oct 23, 2024
@hkershaw-brown hkershaw-brown mentioned this issue Nov 8, 2024
Open
@kdraeder
Copy link
Contributor Author

I've written a script to check the consistency between the dry mass namelist variables (build_dry_mass_pressure_columns and precise_dry_mass_get_close) and the variables in the state vector.
It's in branch cam-se_dry_mass.
It compiles, but I have not set up a test case yet, because I'd like opinions about whether I'm using a good strategy. The strategy is basically to add a function call after static_init_model:call set_cam_variable_info to run get_varid_from_varname for each of the variables which must be in the state vector in order to calculate the dry mass columns. That list is hard-coded in the function, but it could be pulled out to the head of the module. If any of the get_varids fail, the program fails and the user will need to fix the namelist.

@hkershaw-brown
Copy link
Member

Seems like it is best to fail as early as possible and I don't think you can fail any earlier than immediately after reading the what variables are in the state. Its cam-se specific so cam-se model_mod seems like the best place for it (rather than common code)
What's worrying you about this?

@kdraeder
Copy link
Contributor Author

There are usually things I don't understand that others might see immediately.
I don't have much experience using the accessor functions, so I might have picked a suboptimal one.

I looked into putting it into the common code, but the 2 dry_mass namelist variables are not available from the cam-fv namelist. If this is the right solution, I'd put a similar check in cam-fv, but just for PS.

@hkershaw-brown
Copy link
Member

general worry, I dig that.

@kdraeder kdraeder linked a pull request Nov 14, 2024 that will close this issue
Open
15 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@kdraeder kdraeder

@hkershaw-brown hkershaw-brown

Labels
Bug Something isn't working CAM-SE Community Atmosphere Model Spectral Element Core pmp possible Marlee project
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Check list of state variables for required ones NCAR/DART
2 participants
@kdraeder @hkershaw-brown