-
Notifications
You must be signed in to change notification settings - Fork 164
Rewrite of radiance documentation #896
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Thanks for the pull request Lukas! Most of the group is on travel this week and most of next week (I think), so we might be a little slow responding to this. Cheers, |
We appreciate this contribution @lkugler. I will have more time to review in the upcoming weeks, but from skimming through it -- these docs are much improved. |
guide/Radiance_support.rst
Outdated
At present, DART supports RTTOV v12.3 and v13. | ||
In version 12.3, RTTOV-direct for visible/infrared/microwave without | ||
scattering as well as RTTOV-scatt for microwave computations with full | ||
scattering are supported. The interface to v13 is limited to RTTOV-direct. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we need a bit more perspective regarding the limitations of only using RTTOV-direct without full scattering? How much does that deprecate results?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Good that you mention it. It's wrong that RTTOV-direct has no scattering.
see 375bc53
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler As written this states that RTTOVv13 includes the ability to simulate microwave radiance --however the DART interface does not support microwave radiance with v13 because its limited to interacting with RTTOV-direct (UV/VIS/IR) and does not use the RTTOV-scatt (microwave) routine. Is that correct ? Seems like an important point. But looking at the RTTOV obs_def_mod for v12 and v13 there doesn't seem to be any checks or error codes to prevent users from using microwave with RTTOVv13 --- although the DART code still allows for namelist options related to microwave radiance. Could you clarify?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't test the microwave operator before in v13, so I didn't want to claim that it works, sorry.
Now I tested the microwave operator (channel 20 of instrument ATMS on satellite NOAA-21) and noticed that for microwave in v13, the input variables were not using the indices lvlidx
which flip the arrays if user chooses to, i.e. when first_lvl_is_sfc=T
. Moreover, in the hydro(nlevels, ncategories)
array, the order of hydrometeor categories was not correct. Currently it is Rain, Snow, Graupel, Cloud Liquid and Cloud Ice, in that order (says the user guide and the table I used (hydrotable_noaa_atms.dat). However, it depends on the hydrotable and thus might change in the future.
see edd2691, 2886faf, cb09fae
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Exactly, for models that use first_lvl_is_sfc=.false.
. Are there any models that use this convention? I think ECMWF-IFS and related ones maybe.
I think commit cb09fae is more important because it applies for potential WRF/MPAS microwave users.
Something similar was fixed in 2022 (#365 ). Before that, arrays for VIS/IR were flipped twice. Once when the cloud variables are summed up into totalwater/totalice and once when they were assigned to the RTTOV input array runtime % profiles(imem) % cloud(:,:)
. Obviously arrays need to be flipped only once if first_lvl_is_sfc=T.
To compile DART with RTTOV, perform the following steps: | ||
|
||
|
||
**1. Install RTTOV** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
These edits have removed a separate guide for compiling and installing DART-RTTOV 'Getting started with DART RTTOV' originally created by Tim Hoar. Not sure if there is anything worthwhile to retain in those instructions, or whether new steps are enough. Maybe Helen can add some perspective there.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think the edits by Lukas are a great improvement. They give specific instructions for the DART part of the process direct the user to external (RTTOV) documentation when needed.
I am happy to to remove the link to https://github.com/NCAR/DART/wiki/Getting-Started-with-DART-RTTOV since this is out of date (quickbuild.csh).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
EDIT: I will include Tim's instructions on the page, as I found them helpful now while I was doing a fresh install.
Moreover, I'd point to the quick start beginners guide in case our docs become outdated at some point.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Tim's instructions look good incorporated in the High-Level workflow section -- I will clean up the syntax a bit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in f227bc1
guide/Radiance_support.rst
Outdated
* The Deff scheme (`clw_scheme=2`) computes optical properties from an effective particle diameter as input. | ||
By default, DART accesses the model state variable associated with ``QTY_CLOUDWATER_DE`` in the DART namelist. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Technically the default obs_def_rttov_mod code currently in the DART repository prescribes a fixed effective diameter value -- and does not use the QTY_CLOUDWATER_DE value. Should the opposite be true?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
In v13, the code defines a default size. In v12 it doesn't.
I added the default in v12 too now: 4d671f2
Moreover, I added the necessary code such that the particle size can be read from WRF: b0adf4c
However, it might be better NOT to have it as default (to read particle sizes from any model), as I only checked it for WRF and it's likely that you'll run into errors in other models. Also, depending on the configuration, the variable is not in WRF output.
If we don't want to read from file as default, then we only need to comment out the interpolate()
command in obs_def_rttov_mod.f90
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree that we make the prescribed/reasonable particle size as the default option, but If DART finds the QTY_CLOUDWATER_DE value within the DART state, it assumes the user wants to use the model liquid particle size as input and will perform the interpolate() and overwrite the default values. The ice particle size is determined by the 'use_icede' namelist option false = use prescribed/fixed value, true= use model value from QTY_CLOUD_ICE_DE. If using the modeled particle diameter (from WRF) I think we still need the meter to micron units conversion as well. I will have a go at this.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in 71e8ef1
guide/Radiance_support.rst
Outdated
A good overview over the most important parameters for the radiative transfer | ||
can be found in the RTTOV user guide section "Simulation of UV, visible and IR cloud-affected radiances". |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I actually couldn't find this section in the RTTOV user guide --- did the sections change with the updated RTTOV version, or did I just miss it?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah, that's the v14 guide, which changed a lot. The guide for v12 and v13 is quite hidden now.
Is it ok to add links to the pages where v12/v13 are still available?
5bbbe46
in cloud phase (ice versus water) makes a much larger difference. | ||
|
||
https://github.com/NCAR/DART/wiki/Getting-Started-with-DART-RTTOV | ||
#. Ice clouds |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Why call out Ice clouds specifically, if there is not specific information given in this doc? Can we provide similar scientific guidance for Ice clouds similar to liquid clouds above?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, there was something missing. See 282ed10
something else
There in the docs, I mentioned another feature that would be great for anyone comparing with real observations:
One should only count 10% of "snow" towards ice. It sounds like a ad-hoc estimate, but it's a good estimate, supposedly.
It would be good if we could add that multiplier to snow (
totalice(:) = totalice(:) + max(clouds % snow(imem,:),0.0_r8) |
If not implemented, then at least it's important to make new users aware of that.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It seems it would be straightforward to implement this by using the logical 'is_vis' option. If true apply the 0.1 multiplier, if false keep at 1.0.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Addressed in 9aa7dcf
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler Your suggested changes were much needed. Could you take a look at my text changes, and make sure this is consistent with your understanding. Also -- could you take a look at my specific questions, where I am looking for more scientific feedback.
! TODO: specify cfrac (scalar?!) | ||
! runtime % cld_profiles(imem) % cfrac = ? not implemented | ||
! Custom cfrac = vertical maximum of the cloud fraction | ||
runtime % cld_profiles(imem) % cfrac = max(min(maxval(clouds % cfrac(imem,lvlidx)), 1.0_r8), 0.0_r8) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Before, this was not specified. Now I set this microwave-related (scalar) cloudfraction to the vertical maximum cloud fraction in a column. I'm not sure if this is correct.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler I think using the the maximum cloud fraction of the vertical profile taken from WRF as a proxy for the effective cloud fraction (cfrac) required by RTTOV is a good estimate. The scalar 'effective cloud fraction' being the cloud fraction coverage across the ground. The way I see it is that the effective cloud fraction could be no greater than the maximum vertical profile cloud fraction. Depending on grid resolution the cloud fraction will be binary in most cases, I think -- either 0 or 1, so if cloud fraction is 1 at any point in profile, the effective cloud fraction is 1. Seems like this will work.
runtime % cld_profiles(imem) % hydro_frac(:,:) = 1.0_jprb | ||
! runtime % cld_profiles(imem) % hydro_frac(:,:) = clouds % cfrac(imem,lvlidx) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler This seems incorrect. If the cfrac is defined then hydro_frac is assumed to equal cfrac. Can you confirm?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
By the user guide, hydro_frac(nlevels, nhydro_frac)
describes cloud fraction (0-1) on nlevels
for nhydro_frac
many hydrometeor categories.
If we assume nhydro_frac=1
, then we can set hydro_frac(:,1) = cfrac(:)
since we don't have any other information on the spatial extent of rain/snow/graupel/hail within a grid box.
So I think that we can have
runtime % cld_profiles(imem) % hydro_frac(:,1) = clouds % cfrac(imem,lvlidx)
, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler Yes -- I think this is correct and we can switch the line that is commented out such that we specify vertical cloud fraction across all hydrometeors as : <runtime % cld_profiles(imem) % hydro_frac(:,1) = clouds % cfrac(imem,lvlidx)
.
The nhydro_frac
is already defined as 1 during the rttov_alloc_scatt_prof
call within obs_def_rttov13
, so the dimensions should work. I don't see how we could pry any more information out of WRF to specify a per hydrometeor cloud fraction? On a subgrid scale WRF isn't capable of providing this information I wouldn't think.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes. We can assume that as long as we put a note into the documentation.
|
||
! code proposed, depends on the hydrotables of RTTOV? | ||
! TODO: adapt to hydrotable, change indices of hydro(:,X) <--- here | ||
! This code may depend on hydrotables of RTTOV |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler For RTTOV13, however, it looks like the hydrotable order is fixed as you have defined from RTTOV13 users guide:
"Number of hydrometeor types considered by RTTOV-SCATT. The correct number of hydrometeors, and the order in which they are specified, is controlled by the hydrotable*.dat coefficient files. Using standard hydrotables obtained from the NWP SAF website, RTTOVSCATT will expect 5 hydrometeors (rain, snow, graupel, cloud water, cloud ice, in that order), but hydrotables can contain properties for any number of hydrometeor types. "
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Note to self -- Looks like with latest commits WRF-DART RTTOV13 now compatible with microwave emission. Test this and then update docs to reflect this update.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Would you like my model data or namelists to create a test case?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@lkugler What I have been doing thus far is performing a WRF-DART PMO with microwave observations (EOS_AMSU-A) to test out this functionality. Derecho is down the next few days, but it would be nice to have a test case for microwave obs, especially with a wrf simulation where you output cloud fraction (CFRAC). Currently I don't have a wrf run that outputs CFRAC, which would be necessary to test some of the new functionality we have added.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I have completed my testing for the microwave RTTOV13 functionality. I used EOS_2_AMSUA_TB observations and picked locations of cloudy and clear conditions and this seemed to give reasonable BT values. My tests can be found here: /glade/work/bmraczka/DART/models/wrf/MW_rttov13_fix. I used perfect_model_obs
to generate synthetic obs from a wrfinput file based on the CONUS region, with the objective to exercise the new RTTOV13 microwave functionality with a variety of options. I performed two alternative test cases which used empirical formulations for effective hydrometeor radius and cloud fraction, and also prescribed hydrometeor radius and cloud fraction provided from WRF. Roughly speaking these were the namelist settings for each case:
empirical DE &obs_def_rttov
settings:
cfrac_data = .false.
clw_data = .true.
rain_data = .true.
ciw_data = .false.
snow_data = .false.
graupel_data = .false.
hail_data = .false.
..
..
clw_scheme = 1
use_icede = .false.
prescribed DE &obs_def_rttov
settings:
cfrac_data = .true.
clw_data = .true.
rain_data = .true.
ciw_data = .true.
snow_data = .true.
graupel_data = .true.
hail_data = .false.
..
..
clw_scheme = 2
use_icede = .true.
@lkugler If you had a test case you wanted to run through, please go ahead, otherwise I think this is ready to go.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@braczka Perfect. Thank you!
is_vis logical needed to be defined
Also provide more info on prescribing hydrometeor effective radius for microwave calculation
Hi all, just checking in on this radiance documentation pull request #896. Is this ready to me merged and released? |
It's ready to be released, right @braczka ? |
if (allocated(clouds % snow)) then | ||
totalice(:) = totalice(:) + max(clouds % snow(imem,lvlidx),0.0_r8) | ||
! Following Kostka et al., 2014 | ||
if (is_vis) then |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
is_vis is not defined in obs_def_rttov_mod.f90 so this module does not compile:
/glade/derecho/scratch/hkershaw/DART/pull_reqeuests/pull_896/DART/observations/forward_operators/obs_def_mod.f90(1914): error #6404: This name does not have a type, and must have an explicit type. [IS_VIS]
if (is_vis) then
Does this need is_vis defined the same way as rttov13:
is_vis = sensor%sensor_type_name == 'vis'
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I think I only applied this commit to obs_def_rttov_mod13.f90 in order to create is_vis, but probably should be applied to obs_def_rttov_mod.f90 as well ??
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This commit: 817cf37
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It does look like the is_vis line was just missed from obs_def_mod_rttov.f90. @braczka Do you want me to go ahead and make the change to add in is_vis = sensor%sensor_type_name == 'vis'
?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I cannot recreate the build error -- I can successfully build on derecho for this pull request using template mkmf.template.rttov.gfortran for RTTOV 13 using:
RTTOV = /glade/campaign/cisl/dares/libraries/rttov132_gfortran/
RTTOV_VERSION = 13
with the preprocess_nml pointing to:
obs_type_files = '../../../observations/forward_operators/obs_def_rttov13_mod.f90',
and I can successfully build RTTOV 12 using:
RTTOV = /glade/campaign/cisl/dares/libraries/rttov123_gfortran/
RTTOV_VERSION = 12
with the preprocess_nml pointing to:
obs_type_files = '../../../observations/forward_operators/obs_def_rttov_mod.f90',
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@braczka which branch are you building, Lukas's branch (this pull request) or NCAR/radiance-docs where I fixed the compile issue?
main...radiance-docs
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I guess it was the NCAR/radiance-docs because now that I look at the obs_def_rttov_mod.f90 it has the is_vis changes -- didn't realize those changes had been implemented given your last comment.
Regardless - this seems to be resolved and so just go ahead as you see fit.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The changes haven't been implemented on this pull request. Its two different branches.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you give a summary of the code changes for the release notes, the description in the pull request only has the documentation rewrite.
Thanks,
Helen
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I don't have access to a computer so just popping notes in here:
Enables microwave radiance emission for RTTOV13
-
Allows user to bring in liquid/ice effective droplet radius from model
as alternative to the fixed default value -
Specifies the max cloudfraction from the model as the
effective cloud fraction -
Fixes hydrometeor type 'hydrotable' to expect the order: rain, snow, graupel, cloud water
cloud ice. -
For visible radiance applies empirical 0.1 factor against snow when calculating
total cloud ice
Contributed with help from Lukas Kugler
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
leaving this on request changes, obs_def_rttov_mod.f90 does not not compile
export RTTOV=${RTTOV_install} | ||
cd $DART/models/wrf/work/ | ||
ln -sf $DART/observations/forward_operators/rttov_sensor_db.csv . | ||
ln -sf $RTTOV/rtcoef_rttov13/cldaer_visir/sccldcoef_msg_4_seviri.dat . |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
rtcoef_rttov13, however the build instructions are for rttov12 (obs_def_rttov_mod.f90)
DART can be built with either RTTOV v12 *or* v13. Edit :ref:`&preprocess_nml <preprocess>` to select | ||
the appropriate obs_def: | ||
|
||
- obs_def_rttov_mod.f90 for v12.3 | ||
- obs_def_rttov13_mod.f90 for v13 (contributed by Lukas Kugler of the University of Vienna). | ||
|
||
Note the namelist options for &obs_def_rttov_nml differ for v12 and v13. | ||
|
||
- `RTTOV v12 Namelist`_ &obs_def_rttov_nml | ||
- `RTTOV v13 Namelist`_ &obs_def_rttov_nml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How to compile with rttov v12 or v13 seems to be missing from the new documentation, unless I'm missing something. This seems pretty important info for compiling DART with RTTOV.
Co-authored-by: Helen Kershaw <[email protected]>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Accepted all precision changes, and suggesting to add back instructions on RTTOV v12 versus v13 compile.
|
||
radiances/brightness temperatures. Observations from a wide range of satellites (e.g. GOES, FY, METOP, ...) and | ||
sensors (e.g. ABI, AMSU-A, SEVIRI, ...) are supported | ||
(`see the complete list here <https://nwp-saf.eumetsat.int/site/software/rttov/documentation/platforms-supported/>`__). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
(`see the complete list here <https://nwp-saf.eumetsat.int/site/software/rttov/documentation/platforms-supported/>`__). | |
(`see the complete list here <https://nwp-saf.eumetsat.int/site/software/rttov/documentation/platforms-supported/>`__). | |
DART can be built with either RTTOV v12 *or* v13. Edit :ref:`&preprocess_nml <preprocess>` to select | |
the appropriate obs_def: | |
- obs_def_rttov_mod.f90 for v12.3 | |
- obs_def_rttov13_mod.f90 for v13 | |
Note the namelist options for &obs_def_rttov_nml differ for v12 and v13. | |
- `RTTOV v12 Namelist`_ &obs_def_rttov_nml | |
- `RTTOV v13 Namelist`_ &obs_def_rttov_nml |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@hkershaw-brown I agree that we want to still include instructions for v12.3 and v13. I suggest adding back these instructions here to toggle between obs_def_rttov_mod.f90 and obs_def_rttov13_mod.f90 in preprocess namelist.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Description:
I think the radiance documentation needed some rewriting.
Please feel free to suggest additional changes / what's missing.
I think it shouldn't be a duplicate of the RTTOV user guide, but give an overview over capabilities, list advice, and give references to further material.
Fixes issue
None
Types of changes
Documentation changes needed?
Tests
You can view the docs at readthedocs:
Radiance_support
obs-def-rttov-mod
Checklist for merging
Checklist for release
Testing Datasets