Skip to content

Commit f193ab4

Browse files
Merge pull request #708 from NCAR/WRF_forward_operators
Wrf forward operators
2 parents 75cf8dc + b5e8ac1 commit f193ab4

File tree

8 files changed

+696
-424
lines changed

8 files changed

+696
-424
lines changed

CHANGELOG.rst

+9
Original file line numberDiff line numberDiff line change
@@ -22,6 +22,15 @@ individual files.
2222

2323
The changes are now listed with the most recent at the top.
2424

25+
**August 15 2024 :: WRF fwd operator bug fixes. Tag v11.6.1**
26+
27+
WRF-DART bug-fixes:
28+
29+
- Bug fix for surface temperature observations to use QTY_2M_TEMPERATURE
30+
- Bug fix for conversion of vapor mixing ratio to specific humidity
31+
- Bug fix for diagnostics_obs.csh
32+
- Improved documentation for WRF model_mod and WRF-DART Tutorial
33+
2534
**July 26 2024 :: Library build tools for DART. Tag v11.6.0**
2635

2736
- Buildtools for compiling DART as a shared or a static library.

conf.py

+1-1
Original file line numberDiff line numberDiff line change
@@ -21,7 +21,7 @@
2121
author = 'Data Assimilation Research Section'
2222

2323
# The full version, including alpha/beta/rc tags
24-
release = '11.6.0'
24+
release = '11.6.1'
2525
root_doc = 'index'
2626

2727
# -- General configuration ---------------------------------------------------

models/wrf/model_mod.f90

+103-86
Large diffs are not rendered by default.

models/wrf/readme.rst

+335-306
Large diffs are not rendered by default.

models/wrf/shell_scripts/driver.csh

+6-7
Original file line numberDiff line numberDiff line change
@@ -473,19 +473,18 @@ while ( 1 == 1 )
473473

474474
else if ( $SUPER_PLATFORM == 'derecho' ) then
475475

476-
# Prevent double submission for member 1 only
477-
if ( $n == 1) then
478-
sleep 5
479-
endif
480476

481477
if ( `qstat -wa | grep assim_advance_${n} | wc -l` == 0 ) then
482478

483-
echo "assim_advance_${n} is missing from the queue"
484-
qsub assim_advance_mem${n}.csh
479+
echo "Warning, detected that assim_advance_${n} is missing from the queue"
480+
echo "If this warning leads to missing output from ensemble ${n}"
481+
echo "consider enabling the qsub command within keep_trying while statement in driver.csh"
482+
483+
#qsub assim_advance_mem${n}.csh
485484
endif
486485

487486
endif
488-
sleep 15
487+
sleep 5
489488

490489
end
491490
set start_time = `head -1 start_member_${n}`

models/wrf/shell_scripts/mean_increment.ncl

+8-9
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
; find the mean state space increment, output the fields to a single mean file
22
; that can be used to make plots
33
; G. Romine 2011-12
4-
4+
; Updating for 1 domain only B. Raczka 2024-08
55
begin
66

77
; get the list of files to read in
@@ -23,9 +23,7 @@ begin
2323
ListSetType(fil, "join")
2424

2525
pull_2D_field_names = (/"T2", "Q2", "U10", "V10", "PSFC"/)
26-
pull_2D_field_names(:) = pull_2D_field_names(:)+"_d01"
27-
pull_3D_field_names = (/"U", "V", "T", "QVAPOR"/)
28-
pull_3D_field_names(:) = pull_3D_field_names(:)+"_d01"
26+
pull_3D_field_names = (/"U", "V", "THM", "QVAPOR"/)
2927
npulls = dimsizes(pull_2D_field_names)
3028

3129
; Below will dump out the data to a file for replotting later
@@ -34,16 +32,17 @@ begin
3432
do i=0,npulls-1
3533
print(" Extracting 2d variable "+pull_2D_field_names(i))
3634
do fil_num=0,nfils-1
37-
; print(" reading file "+flist(fil_num))
38-
; dimensions are ncljoin, copy, Time, south_north, west_east
35+
; print(" reading file "+flist(fil_num))
36+
; dimensions are ncljoin, Time, south_north, west_east
3937
; copy zero is the ensemble mean
40-
pull_var = fil[fil_num]->$pull_2D_field_names(i)$(:,0,:,:,:)
38+
pull_var = fil[fil_num]->$pull_2D_field_names(i)$(:,:,:,:)
4139
dims = dimsizes(pull_var)
4240
if (fil_num .eq. 0) then ; first iteration, make var
4341
alltimes_var = new ( (/nfils,dims(2),dims(3)/), typeof(pull_var) )
4442
end if
4543
; printVarSummary(pull_var)
4644
alltimes_var(fil_num,:,:) = pull_var(0,0,:,:)
45+
; printVarSummary(alltimes_var)
4746
delete(pull_var)
4847
end do
4948
; average over time (first dimension)
@@ -69,9 +68,9 @@ begin
6968
print(" Extracting 3d variable "+pull_3D_field_names(i))
7069
do fil_num=0,nfils-1
7170
; print(" reading file "+flist(fil_num))
72-
; dimensions are ncljoin, copy, Time, level, south_north, west_east
71+
; dimensions are ncljoin, Time, level, south_north, west_east
7372
; copy zero is the ensemble mean
74-
pull_var = fil[fil_num]->$pull_3D_field_names(i)$(:,0,:,:,:,:)
73+
pull_var = fil[fil_num]->$pull_3D_field_names(i)$(:,:,:,:,:)
7574
dims = dimsizes(pull_var)
7675
if (fil_num .eq. 0) then ; first iteration, make var
7776
alltimes_var = new ( (/nfils,dims(2),dims(3),dims(4)/), typeof(pull_var) )

models/wrf/tutorial/README.rst

+228-8
Original file line numberDiff line numberDiff line change
@@ -600,7 +600,7 @@ also want to modify this script to test running a single member first —
600600
just in case you have some debugging to do.
601601

602602
However, be warned that to successfully complete the tutorial, including
603-
running the *driver.csh* script in Step 5, using a smaller ensemble
603+
running the *driver.csh* script in Step 6, using a smaller ensemble
604604
(e.g. < 20 members) can lead to spurious updates during the analysis step,
605605
causing the WRF simulation to fail.
606606

@@ -617,7 +617,7 @@ Step 3: Prepare observations [OPTIONAL]
617617

618618
The observation sequence files to run this tutorial are already provided
619619
for you. If you want to run with the provided tutorial observations, you
620-
can skip to Step 4 right now. If you are interested in using custom
620+
can skip to Step 5 right now. If you are interested in using custom
621621
observations for a WRF experiment other than the tutorial you should read on.
622622
The remaining instructions provided below in Step 3 are meant as a guideline
623623
to converting raw PREPBUFR data files into the required ``obs_seq`` format
@@ -807,8 +807,228 @@ you would do the following:
807807
namelist options to consider, and you must provide a *wrfinput* file
808808
for the program to access the analysis domain information.
809809

810+
Step 4: Overview of forward (observation) operators [OPTIONAL]
811+
--------------------------------------------------------------
810812

811-
Step 4: Creating the first set of adaptive inflation files
813+
This section is for informational purposes only and does not include any
814+
instructions to complete the tutorial. It provides a description of
815+
the DART settings that control the forward operator which
816+
calculates the prior and posterior model estimates for the observations.
817+
An introduction to important namelist variables that control the operation of the forward
818+
operator are located in the `WRF namelist documentation.
819+
<../../../models/wrf/readme.html#namelist>`__
820+
821+
822+
The ``obs_seq.out`` file generated as described in Step 3 includes a total
823+
of 30 observation types. Here we examine an exerpt of that file, focusing
824+
on a single temperature observation to describe the process:
825+
826+
::
827+
828+
obs_sequence
829+
obs_kind_definitions
830+
30
831+
41 METAR_TEMPERATURE_2_METER
832+
..
833+
..
834+
num_copies: 1 num_qc: 1
835+
num_obs: 70585 max_num_obs: 70585
836+
NCEP BUFR observation
837+
NCEP QC index
838+
first: 1 last: 70585
839+
OBS 1
840+
288.750000000000
841+
1.00000000000000
842+
-1 2 -1
843+
obdef
844+
loc3d
845+
4.819552185804497 0.6141813398083548 518.0000000000000 -1
846+
kind
847+
41
848+
43200 152057
849+
3.06250000000000
850+
..
851+
..
852+
..
853+
854+
855+
A critical piece of observation metadata includes the observation type
856+
(``METAR_TEMPERATURE_2_METER``) which is linked to the quantity
857+
(``QTY_2M_TEMPERATURE``) through the observation definition file
858+
(``obs_def_metar_mod.f90``). This file is included within the
859+
``&preprocess_nml`` section of the namelist file as:
860+
861+
::
862+
863+
&preprocess_nml
864+
overwrite_output = .true.
865+
input_obs_qty_mod_file = '../../../assimilation_code/modules/observations/DEFAULT_obs_kind_mod.F90'
866+
output_obs_qty_mod_file = '../../../assimilation_code/modules/observations/obs_kind_mod.f90'
867+
input_obs_def_mod_file = '../../../observations/forward_operators/DEFAULT_obs_def_mod.F90'
868+
output_obs_def_mod_file = '../../../observations/forward_operators/obs_def_mod.f90'
869+
quantity_files = '../../../assimilation_code/modules/observations/atmosphere_quantities_mod.f90'
870+
obs_type_files = '../../../observations/forward_operators/obs_def_reanalysis_bufr_mod.f90',
871+
'../../../observations/forward_operators/obs_def_altimeter_mod.f90',
872+
'../../../observations/forward_operators/obs_def_radar_mod.f90',
873+
'../../../observations/forward_operators/obs_def_metar_mod.f90',
874+
..
875+
..
876+
..
877+
878+
During the DART compilation described within Step 1 this information is
879+
included within the ``obs_def_mod.f90``.
880+
881+
The vertical coordinate type is the 4th column beneath the loc3d header within ``obs_seq.out``.
882+
In this example the value -1 indicates the vertical coordinate is ``VERTISSURFACE``. It defines the
883+
vertical units of the observation (e.g. pressure, meters above sea level, model levels etc).
884+
This serves two purposes -- foremost it is required during the vertical spatial interpolation
885+
to calculate the precise location of the expected observation.
886+
A second crtical function is that it defines whether it is a surface observation.
887+
Observations with a vertical coordinate of ``VERTISSURFACE`` are defined as surface
888+
observations. All other coordinates are considered non-surface observations
889+
(e.g. profile observations). Of note is that the vertical coordinate ``VERTISSURFACE`` and
890+
``VERTISHEIGHT`` are functionally identical (i.e. meters above sea level), however
891+
only the ``VERTISSURFACE`` is a surface observation.
892+
893+
For more information on the vertical coordinate metadata see the detailed structure of
894+
an `obs_seq file. <../../../guide/creating-obs-seq-real.html#observation-location>`__
895+
896+
In order to connect this observation to the appropriate WRF output variables
897+
the ``wrf_state_variables`` within ``&model_nml`` defines the *WRF field name* and
898+
the *WRF TYPE* in the 1st and 3rd columns as shown in the tutorial example below:
899+
900+
::
901+
902+
&model_nml
903+
wrf_state_variables = 'T2','QTY_TEMPERATURE','TYPE_T2','UPDATE','999'
904+
905+
..
906+
..
907+
908+
For more information on the ``&model_nml`` variables see the `WRF documentation page
909+
<../../../models/wrf/readme.html#namelist>`__
910+
911+
912+
As described above, the linkage between the observation type and the WRF output field
913+
is defined through the physical quantity, surface variable designation (observation
914+
vertical coordinate), and WRF TYPE. The current design of the WRF ``model_mod.f90``
915+
is such that the quantity is a general classification (e.g. temperature, wind
916+
specific humidity), whereas the WRF TYPE classification is more precisely
917+
mapped to the WRF output field. The table below summarizes the dependency between
918+
the observation type and the WRF output field for a select number of observation types
919+
within the tutorial.
920+
921+
.. Note::
922+
923+
The number of WRF output fields required to support an observation type can vary. For
924+
observation types where there is a direct analog to a WRF output field, the forward
925+
operator consists of only spatial interpolation, thus requires only a single output
926+
variable (e.g. METAR_TEMPERATURE_2_METER). For observation types that require multiple
927+
WRF output fields, the forward operator is more complex than a simple spatial interpolation.
928+
For more information see the notes below the table. A rule of thumb is a surface
929+
observation should use a surface output field (e.g. T2, U10). WRF surface output fields
930+
are appended by a numeric value indicating surface height in meters. It is possible to use
931+
a non-surface WRF output field (3D field) to estimate a surface observation, however, this
932+
requires a vertical interpolation of the 3D WRF field where the observed surface height does
933+
not coincide with the model levels. This either requires a vertical interpolation or an
934+
extrapolation which can be **inaccurate and is not recommended**.
935+
936+
937+
938+
939+
+----------------------------------+---------+-------------------------------+--------------+------------+
940+
| DART Observation Type | Surface | DART Quantity | WRF Type | WRF output |
941+
| | Obs ? | | | field |
942+
+==================================+=========+===============================+==============+============+
943+
| ``METAR_TEMPERATURE_2_METER`` | Yes | ``QTY_2M_TEMPERATURE`` | ``TYPE_T2`` | ``T2`` |
944+
| | | | | |
945+
+----------------------------------+---------+-------------------------------+--------------+------------+
946+
| ``RADIOSONDE_TEMPERATURE`` | No | ``QTY_POTENTIAL_TEMPERATURE`` | ``TYPE_T`` | ``THM`` |
947+
| | | ``QTY_VAPOR_MIXING_RATIO`` | ``TYPE_QV`` | ``QVAPOR`` |
948+
| | | ``QTY_PRESSURE`` | ``TYPE_MU`` | ``MU PH`` |
949+
| | | ``QTY_GEOPOTENTIAL_HEIGHT`` | ``TYPE_GZ`` | |
950+
+----------------------------------+---------+-------------------------------+--------------+------------+
951+
| ``METAR_U_10_METER_WIND`` | Yes | ``QTY_U_WIND_COMPONENT`` | ``TYPE_U10`` | ``U10`` |
952+
| | | ``QTY_V_WIND_COMPONENT`` | ``TYPE_V10`` | ``V10`` |
953+
+----------------------------------+---------+-------------------------------+--------------+------------+
954+
| ``ACARS_U_WIND_COMPONENT`` | No | ``QTY_U_WIND_COMPONENT`` | ``TYPE_U`` | ``U`` |
955+
| | | ``QTY_V_WIND_COMPONENT`` | ``TYPE_V`` | ``V`` |
956+
+----------------------------------+---------+-------------------------------+--------------+------------+
957+
| ``METAR_DEWPOINT_2_METER`` | Yes | ``QTY_DEWPOINT`` | | |
958+
| | | ``QTY_SPECIFIC_HUMIDITY`` | ``TYPE_Q2`` | ``Q2`` |
959+
| | | ``QTY_PRESSURE`` | ``TYPE_PS`` | ``PSFC`` |
960+
+----------------------------------+---------+-------------------------------+--------------+------------+
961+
| ``RADIOSONDE_SPECIFIC_HUMIDITY`` | No | ``QTY_SPECIFIC_HUMIDITY`` | ``TYPE_QV`` | ``QVAPOR`` |
962+
| | | | | |
963+
+----------------------------------+---------+-------------------------------+--------------+------------+
964+
965+
966+
967+
Surface Temperature (e.g. METAR_TEMPERATURE_2_METER)
968+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
969+
970+
WRF output includes a direct analog for sensible temperature surface observations (e.g. T2), thus
971+
the forward operator requires only 1 variable to calculate the expected observation.
972+
The calculation includes a horizontal interpolation of the 2D temperature variable (e.g. T2).
973+
974+
975+
Non-Surface Temperature (e.g. RADIOSONDE_TEMPERATURE)
976+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
977+
978+
In contrast to surface temperature observations, non-surface temperature observations require 4 WRF
979+
output fields. This is because observations are sensible temperature, whereas the 3D WRF
980+
temperature field is provided in perturbation potential temperature. Thus, the forward
981+
operator first requires a physical conversion between perturbation potential temperature to
982+
sensible temperature, followed by a spatial interpolation (this includes horizontal interpolation
983+
on WRF levels k and k+1, followed by vertical interpolation).
984+
985+
.. Important::
986+
987+
There are two different 3D temperature WRF output fields that can work to calculate non-
988+
surface temperature observations (e.g. T or THM, T=THM when use_theta_m=0). However, and **of
989+
utmost importance** is the variable THM is required to be within the ``&model_nml`` if the
990+
3D temperature field is to be updated in the ``filter`` step. **This is because the WRF field *T*
991+
is a diagnostic variable with no impact on the forecast step, whereas the WRF field *THM* is
992+
a prognostic field which will impact the forecast.**
993+
994+
995+
Surface Wind (e.g. METAR_U_10_METER_WIND)
996+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
997+
998+
Surface winds have a direct WRF output analog (e.g. U10)
999+
and requires horizontal interpolation of the 2D zonal wind field. However, the
1000+
meridional wind (e.g. V10) is also required in order to convert from modeled *gridded* winds to
1001+
*true* wind observations. This requirement is an artifact of winds measured on a sphere being
1002+
mapped on a 2D grid.
1003+
1004+
1005+
Non-Surface Wind (e.g. ACARS_U_WIND_COMPONENT)
1006+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1007+
1008+
This is identical to surface winds as described above, except the spatial interpolation requires
1009+
horizontal interpolation on the k and k+1 WRF levels, followed by vertical interpolation.
1010+
1011+
1012+
Surface Dewpoint (e.g. METAR_DEWPOINT_2_METER)
1013+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1014+
1015+
The calculation of surface dewpoint requires a physical conversion using both surface
1016+
pressure (PSFC) and surface vapor mixing ratio (Q2), follwed by horizontal interpolation.
1017+
1018+
1019+
Non-Surface Specific Humidity (e.g. RADIOSONDE_SPECIFIC_HUMIDITY)
1020+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
1021+
1022+
Specific humidity observations require the (water) vapor mixing ratio (QVAPOR) for the forward operator.
1023+
Although specific humidity and vapor mixing ratio are nearly identical, especially in dry
1024+
air, they are actually two distinct physical properties -- the ratio of water mass to total air mass
1025+
versus ratio of water vapor mass to dry air mass respectively. Therefore the forward operator
1026+
includes this physical conversion followed by a spatial interpolation (i.e. horizontal interpolation of k and
1027+
k+1 WRF vertical levels followed by vertical interpolation).
1028+
1029+
1030+
1031+
Step 5: Creating the first set of adaptive inflation files
8121032
----------------------------------------------------------
8131033

8141034
In this section we describe how to create initial adaptive inflation
@@ -874,7 +1094,7 @@ cycle.
8741094

8751095

8761096

877-
Step 5: Cycled analysis system
1097+
Step 6: Cycled analysis system
8781098
------------------------------
8791099

8801100
While the DART system provides executables to perform individual tasks
@@ -924,10 +1144,10 @@ continue to cycle until the final analysis time has been reached.
9241144

9251145

9261146

927-
Step 6: Diagnosing the assimilation results
1147+
Step 7: Diagnosing the assimilation results
9281148
-------------------------------------------
9291149

930-
Once you have successfully completed steps 1-5, it is important to
1150+
Once you have successfully completed steps 1-6, it is important to
9311151
check the quality of the assimilation. In order to do this, DART provides
9321152
analysis system diagnostics in both state and observation space.
9331153

@@ -989,7 +1209,7 @@ The tools below provide methods to visualize the spatial patterns, statistics an
9891209
failure mode for all observations.
9901210

9911211
The observation diagnostics use the **obs_epoch*.nc** file as input. This file is
992-
automatically generated by the **obs_diagnostic.csh** script within Step 5 of this
1212+
automatically generated by the **obs_diagnostic.csh** script within Step 6 of this
9931213
tutorial.
9941214

9951215
The **obs_epoch*.nc** file is located in the output directory of each time step.
@@ -1211,7 +1431,7 @@ quite high (>90%). This high acceptance percentage is typical of a high-quality
12111431
assimilation and consistent with the strong reduction in RMSE.
12121432

12131433

1214-
The same plot as above is given below except for the observation type:
1434+
The same plot as above except for the observation type:
12151435
``RADIOSONE_SPECIFIC_HUMIDITY``.
12161436

12171437
+-------------------------------------------------------------+

0 commit comments

Comments
 (0)