Rewrite of radiance documentation #896

lkugler · 2025-06-11T14:28:15Z

Description:

I think the radiance documentation needed some rewriting.
Please feel free to suggest additional changes / what's missing.
I think it shouldn't be a duplicate of the RTTOV user guide, but give an overview over capabilities, list advice, and give references to further material.

Fixes issue

None

Types of changes

Bug fix (non-breaking change which fixes an issue)
New feature (non-breaking change which adds functionality)
Breaking change (fix or feature that would cause existing functionality to not work as expected)
Documentation update

Documentation changes needed?

My change requires a change to the documentation.
- I have updated the documentation accordingly.

Tests

You can view the docs at readthedocs:
Radiance_support
obs-def-rttov-mod

Checklist for merging

Updated changelog entry
Documentation updated
Update conf.py

Checklist for release

Merge into main
Create release from the main branch with appropriate tag
Delete feature-branch

Testing Datasets

Dataset needed for testing available upon request
Dataset download instructions included
No dataset needed

hkershaw-brown · 2025-06-13T16:58:24Z

Thanks for the pull request Lukas! Most of the group is on travel this week and most of next week (I think), so we might be a little slow responding to this.

Cheers,
Helen

braczka · 2025-06-20T19:50:42Z

We appreciate this contribution @lkugler. I will have more time to review in the upcoming weeks, but from skimming through it -- these docs are much improved.

braczka · 2025-06-25T22:42:44Z

guide/Radiance_support.rst

+At present, DART supports RTTOV v12.3 and v13.  
+In version 12.3, RTTOV-direct for visible/infrared/microwave without 
+scattering as well as RTTOV-scatt for microwave computations with full 
+scattering are supported. The interface to v13 is limited to RTTOV-direct.


Do we need a bit more perspective regarding the limitations of only using RTTOV-direct without full scattering? How much does that deprecate results?

Good that you mention it. It's wrong that RTTOV-direct has no scattering.
see 375bc53

@lkugler As written this states that RTTOVv13 includes the ability to simulate microwave radiance --however the DART interface does not support microwave radiance with v13 because its limited to interacting with RTTOV-direct (UV/VIS/IR) and does not use the RTTOV-scatt (microwave) routine. Is that correct ? Seems like an important point. But looking at the RTTOV obs_def_mod for v12 and v13 there doesn't seem to be any checks or error codes to prevent users from using microwave with RTTOVv13 --- although the DART code still allows for namelist options related to microwave radiance. Could you clarify?

I didn't test the microwave operator before in v13, so I didn't want to claim that it works, sorry.
Now I tested the microwave operator (channel 20 of instrument ATMS on satellite NOAA-21) and noticed that for microwave in v13, the input variables were not using the indices lvlidx which flip the arrays if user chooses to, i.e. when first_lvl_is_sfc=T. Moreover, in the hydro(nlevels, ncategories) array, the order of hydrometeor categories was not correct. Currently it is Rain, Snow, Graupel, Cloud Liquid and Cloud Ice, in that order (says the user guide and the table I used (hydrotable_noaa_atms.dat). However, it depends on the hydrotable and thus might change in the future.
see edd2691, 2886faf, cb09fae

@lkugler This commit edd2691 (level ordering) looks like an important fix that applies to all wavebands (NIR/VIS/MW), whereas the other commits (cloudfraction and hydrometeor ordering) apply specifically to MW only. Could you confirm?

Exactly, for models that use first_lvl_is_sfc=.false.. Are there any models that use this convention? I think ECMWF-IFS and related ones maybe.
I think commit cb09fae is more important because it applies for potential WRF/MPAS microwave users.

Something similar was fixed in 2022 (#365 ). Before that, arrays for VIS/IR were flipped twice. Once when the cloud variables are summed up into totalwater/totalice and once when they were assigned to the RTTOV input array runtime % profiles(imem) % cloud(:,:). Obviously arrays need to be flipped only once if first_lvl_is_sfc=T.

braczka · 2025-06-25T22:44:37Z

guide/Radiance_support.rst

+To compile DART with RTTOV, perform the following steps:
+
+
+**1. Install RTTOV**


These edits have removed a separate guide for compiling and installing DART-RTTOV 'Getting started with DART RTTOV' originally created by Tim Hoar. Not sure if there is anything worthwhile to retain in those instructions, or whether new steps are enough. Maybe Helen can add some perspective there.

I think the edits by Lukas are a great improvement. They give specific instructions for the DART part of the process direct the user to external (RTTOV) documentation when needed.
I am happy to to remove the link to https://github.com/NCAR/DART/wiki/Getting-Started-with-DART-RTTOV since this is out of date (quickbuild.csh).

EDIT: I will include Tim's instructions on the page, as I found them helpful now while I was doing a fresh install.
Moreover, I'd point to the quick start beginners guide in case our docs become outdated at some point.

Tim's instructions look good incorporated in the High-Level workflow section -- I will clean up the syntax a bit.

Addressed in f227bc1

braczka · 2025-06-25T22:49:37Z

guide/Radiance_support.rst

+   *  The Deff scheme (`clw_scheme=2`) computes optical properties from an effective particle diameter as input.
+      By default, DART accesses the model state variable associated with ``QTY_CLOUDWATER_DE`` in the DART namelist.


Technically the default obs_def_rttov_mod code currently in the DART repository prescribes a fixed effective diameter value -- and does not use the QTY_CLOUDWATER_DE value. Should the opposite be true?

In v13, the code defines a default size. In v12 it doesn't.
I added the default in v12 too now: 4d671f2

Moreover, I added the necessary code such that the particle size can be read from WRF: b0adf4c
However, it might be better NOT to have it as default (to read particle sizes from any model), as I only checked it for WRF and it's likely that you'll run into errors in other models. Also, depending on the configuration, the variable is not in WRF output.
If we don't want to read from file as default, then we only need to comment out the interpolate() command in obs_def_rttov_mod.f90

I agree that we make the prescribed/reasonable particle size as the default option, but If DART finds the QTY_CLOUDWATER_DE value within the DART state, it assumes the user wants to use the model liquid particle size as input and will perform the interpolate() and overwrite the default values. The ice particle size is determined by the 'use_icede' namelist option false = use prescribed/fixed value, true= use model value from QTY_CLOUD_ICE_DE. If using the modeled particle diameter (from WRF) I think we still need the meter to micron units conversion as well. I will have a go at this.

Addressed in 71e8ef1

braczka · 2025-06-25T22:51:00Z

guide/Radiance_support.rst

+A good overview over the most important parameters for the radiative transfer
+can be found in the RTTOV user guide section "Simulation of UV, visible and IR cloud-affected radiances".


I actually couldn't find this section in the RTTOV user guide --- did the sections change with the updated RTTOV version, or did I just miss it?

Ah, that's the v14 guide, which changed a lot. The guide for v12 and v13 is quite hidden now.
Is it ok to add links to the pages where v12/v13 are still available?
5bbbe46

braczka · 2025-06-25T22:52:20Z

guide/Radiance_support.rst


-https://github.com/NCAR/DART/wiki/Getting-Started-with-DART-RTTOV
+#. Ice clouds


Why call out Ice clouds specifically, if there is not specific information given in this doc? Can we provide similar scientific guidance for Ice clouds similar to liquid clouds above?

Yes, there was something missing. See 282ed10

something else

There in the docs, I mentioned another feature that would be great for anyone comparing with real observations:
One should only count 10% of "snow" towards ice. It sounds like a ad-hoc estimate, but it's a good estimate, supposedly.
It would be good if we could add that multiplier to snow (

DART/observations/forward_operators/obs_def_rttov13_mod.f90

Line 2195 in d908222

totalice(:) = totalice(:) + max(clouds % snow(imem,:),0.0_r8)

) and which is only 0.1 for visible radiances and 1.0 else.
If not implemented, then at least it's important to make new users aware of that.

It seems it would be straightforward to implement this by using the logical 'is_vis' option. If true apply the 0.1 multiplier, if false keep at 1.0.

Addressed in 9aa7dcf

braczka

@lkugler Your suggested changes were much needed. Could you take a look at my text changes, and make sure this is consistent with your understanding. Also -- could you take a look at my specific questions, where I am looking for more scientific feedback.

guide/Radiance_support.rst

… radiance-docs

Co-authored-by: Helen Kershaw <[email protected]>

… radiance-docs

lkugler · 2025-07-04T18:17:57Z

I think the guide is in a good condition now. Please have a look.

Edit: see the web version here.

…n hydro array corrected (microwave)

… radiance-docs

lkugler · 2025-07-10T16:19:29Z

observations/forward_operators/obs_def_rttov13_mod.f90

   do ilvl=1,nlevels
      lvlidx(ilvl) = nlevels - ilvl + 1
   end do
 else
-   do ilvl=nlevels,1,-1
+   ! do nothing, as the input data is ordered from TOA to surface
+   do ilvl=1,nlevels


Before, we had indices n, n-1, n-2, ... in both if clauses. But actually we want 1,2,3,.. here.

lkugler · 2025-07-10T16:20:12Z

observations/forward_operators/obs_def_rttov_mod.f90

   do ilvl=1,nlevels
      lvlidx(ilvl) = nlevels - ilvl + 1
   end do
 else
-   do ilvl=nlevels,1,-1
+   ! do nothing, as the input data is ordered from TOA to surface
+   do ilvl=1,nlevels


Before, we had indices n, n-1, n-2, ... in both IF clauses. But actually we want 1,2,3,.. here.

lkugler · 2025-07-10T16:21:59Z

observations/forward_operators/obs_def_rttov13_mod.f90

-         ! TODO: specify cfrac (scalar?!)
-         ! runtime % cld_profiles(imem) % cfrac = ? not implemented
+         ! Custom cfrac = vertical maximum of the cloud fraction
+         runtime % cld_profiles(imem) % cfrac = max(min(maxval(clouds % cfrac(imem,lvlidx)), 1.0_r8), 0.0_r8)


Before, this was not specified. Now I set this microwave-related (scalar) cloudfraction to the vertical maximum cloud fraction in a column. I'm not sure if this is correct.

braczka · 2025-07-10T18:13:49Z

observations/forward_operators/obs_def_rttov13_mod.f90

+         runtime % cld_profiles(imem) % hydro_frac(:,:) = 1.0_jprb
+         ! runtime % cld_profiles(imem) % hydro_frac(:,:) = clouds % cfrac(imem,lvlidx)


@lkugler This seems incorrect. If the cfrac is defined then hydro_frac is assumed to equal cfrac. Can you confirm?

By the user guide, hydro_frac(nlevels, nhydro_frac) describes cloud fraction (0-1) on nlevels for nhydro_frac many hydrometeor categories.
If we assume nhydro_frac=1, then we can set hydro_frac(:,1) = cfrac(:) since we don't have any other information on the spatial extent of rain/snow/graupel/hail within a grid box.

So I think that we can have
runtime % cld_profiles(imem) % hydro_frac(:,1) = clouds % cfrac(imem,lvlidx), right?

lkugler and others added 8 commits May 22, 2025 16:05

first textual overhaul

601ce5e

rewrite radiance docs

22d0eb2

rewrite

c408464

rewrite

b42cfe1

rewrite

5034e69

rewrite

9514598

link not working

0b67661

Merge branch 'main' into radiance-docs

c676370

braczka self-requested a review June 23, 2025 23:23

braczka added 2 commits June 24, 2025 17:50

Edited RTTOV radiance_support docs

b1f2238

Modify obs_def_rttov docs

9adbf26

braczka reviewed Jun 25, 2025

View reviewed changes

braczka requested changes Jun 25, 2025

View reviewed changes

hkershaw-brown reviewed Jun 30, 2025

View reviewed changes

guide/Radiance_support.rst Outdated Show resolved Hide resolved

hkershaw-brown reviewed Jun 30, 2025

View reviewed changes

guide/Radiance_support.rst Outdated Show resolved Hide resolved

hkershaw-brown reviewed Jun 30, 2025

View reviewed changes

guide/Radiance_support.rst Show resolved Hide resolved

lkugler added 3 commits July 1, 2025 17:57

Merge remote-tracking branch 'refs/remotes/origin/radiance-docs' into…

0e359a2

… radiance-docs

remark about direct/scatt

375bc53

install RTTOV paragraph

abf0c38

lkugler force-pushed the radiance-docs branch from 8edbf5b to abf0c38 Compare July 3, 2025 16:03

lkugler added 4 commits July 4, 2025 17:48

particle diameters from WRF

b0adf4c

default diameter in v12

4d671f2

link to v12/v13 guides

5bbbe46

added info about ice clouds

282ed10

lkugler and others added 7 commits July 4, 2025 19:35

remove ``

632b62f

Co-authored-by: Helen Kershaw <[email protected]>

Update guide/Radiance_support.rst

8dac4a6

Co-authored-by: Helen Kershaw <[email protected]>

add index "radiance"

d028f06

Co-authored-by: Helen Kershaw <[email protected]>

add Tim's install instructions

e2fd1a2

Merge remote-tracking branch 'refs/remotes/origin/radiance-docs' into…

5e9e389

… radiance-docs

overlooked includes to preprocess

ac407bc

format references

3ace80e

braczka and others added 9 commits July 9, 2025 11:39

Radiance docs edits

f227bc1

Modify RTTOV effective particle diameter

71e8ef1

Merge remote-tracking branch 'upstream/main' into radiance-docs

de9451b

Modify snow scatter cross-section

9aa7dcf

radiance doc syntax

ff4571f

arrays were wrongly flipped if first_lvl_is_sfc=F

edd2691

define microwave cfrac

2886faf

did not flip arrays if requested by the user; order of hydrometeors i…

cb09fae

…n hydro array corrected (microwave)

Merge remote-tracking branch 'refs/remotes/origin/radiance-docs' into…

88d9553

… radiance-docs

lkugler commented Jul 10, 2025

View reviewed changes

braczka reviewed Jul 10, 2025

View reviewed changes

		To compile DART with RTTOV, perform the following steps:


		1. Install RTTOV

		* The Deff scheme (`clw_scheme=2`) computes optical properties from an effective particle diameter as input.
		By default, DART accesses the model state variable associated with ``QTY_CLOUDWATER_DE`` in the DART namelist.

		A good overview over the most important parameters for the radiative transfer
		can be found in the RTTOV user guide section "Simulation of UV, visible and IR cloud-affected radiances".


		https://github.com/NCAR/DART/wiki/Getting-Started-with-DART-RTTOV
		#. Ice clouds

		runtime % cld_profiles(imem) % hydro_frac(:,:) = 1.0_jprb
		! runtime % cld_profiles(imem) % hydro_frac(:,:) = clouds % cfrac(imem,lvlidx)

Rewrite of radiance documentation #896

Are you sure you want to change the base?

Rewrite of radiance documentation #896

Uh oh!

Conversation

lkugler commented Jun 11, 2025

Description:

Fixes issue

Types of changes

Documentation changes needed?

Tests

Checklist for merging

Checklist for release

Testing Datasets

Uh oh!

hkershaw-brown commented Jun 13, 2025

Uh oh!

braczka commented Jun 20, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkugler Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkugler Jul 10, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkugler Jul 3, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkugler Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkugler Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

lkugler Jul 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

something else

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

braczka left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

lkugler Jul 3, 2025 •

edited

Loading

lkugler Jul 10, 2025 •

edited

Loading

lkugler Jul 3, 2025 •

edited

Loading

lkugler Jul 4, 2025 •

edited

Loading

lkugler Jul 4, 2025 •

edited

Loading

lkugler Jul 4, 2025 •

edited

Loading

lkugler commented Jul 4, 2025 •

edited

Loading

lkugler Jul 10, 2025 •

edited

Loading