Skip to content

CAL‐DISP Interface Acceptance Testing Instructions

Jump to bottom
Riley Kuttruff edited this page Jan 22, 2026 · 3 revisions

This page contains instructions for performing Acceptance Testing for the CAL-DISP Interface delivery from the OPERA-ADT team. These instructions pertain to the latest version of the Interface release, currently v0.1. These instructions assume the user has access to the JPL FN-Artifactory, and has Docker installed on their local machine.

For the current delivery, either Intel or AMD hardware may be used.

Acquiring the CAL-DISP Interface Docker Image

The image is currently hosted on JPL FN-Artifactory, which requires JPL VPN access and JPL credentials. You may also need to be added to the gov.nasa.jpl.opera.adt organization.

Once you have access, the container tarball delivery is available under general/gov/nasa/jpl/opera/adt/cal-disp/r1/interface/dockerimg_cal-disp_interface_0.1.tar . Sample inputs and outputs are also available under general/gov/nasa/jpl/opera/adt/cal-disp/r1/interface/delivery_data_cal-disp_interface_0.1.tar.

Documentation for the delivery is under general/gov/nasa/jpl/opera/adt/cal-disp/r1/interface/documents/

Download both images to a location on your local machine. This location will be referred to throughout this instructions as <CAL-DISP_DIR> Note that the sample data is ~4.7 gigabytes and the image tarball is ~6.7 GB, so the download from AF may take some time.

Loading the image into Docker

The first step in running the CAL-DISP image is to load it into Docker via the following command:

docker load -i <CAL-DISP_DIR>/dockerimg_cal-disp_interface_0.1.tar

This should add the Docker image to your local repository with the name opera/cal-disp and the tag interface_0.1.

Preparing the test data

Once the delivery_data_cal-disp_interface_0.1.tar file is downloaded to your local machine, unpack it to <CAL-DISP_DIR>:

tar -xvf delivery_data_cal-disp_interface_0.1.tar

This will create a delivery_data_cal_disp directory within <CAL-DISP_DIR> containing the following directories:

  • configs/
  • input_data/
    • disp/
    • static_input/
    • tropo/
    • unr/
  • golden_output/

The current delivery is structured so all that needs to be mounted in is <CAL-DISP_DIR> itself without needing to make any extra directories or modifications.

Executing the CAL-DISP container on the sample datasets

We're ready to execute the SAS right away:

docker run --rm --user $(id -u):$(id -g) \
    --volume $(pwd):/home/work \
    --workdir /home/work \
    opera/cal-disp:interface_0.1 cal-disp run delivery_data_cal_disp/configs/runconfig.yaml

The container run time is extremely quick (~30 seconds), with the following output:

2026-01-22 17:49:30,321 - cal_disp - INFO - Output product: /home/work/delivery_data_cal_disp/output/OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260122T174924Z.nc
2026-01-22 17:49:32,308 - cal_disp - INFO - Browse image: /home/work/delivery_data_cal_disp/output/OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260122T174924Z.png
2026-01-22 17:49:32,308 - cal_disp - INFO - cal_disp version: 1.0.0
2026-01-22 17:49:32,308 - cal_disp - INFO - Maximum memory usage: 3.63 GB
Calibration Workflow Configuration
======================================================================

Directories:
  Work directory:   /home/work/delivery_data_cal_disp
  Output directory: /home/work/delivery_data_cal_disp/output
  Log file:         /home/work/delivery_data_cal_disp/cal_disp.log
  Keep relative:    False

Input Files:
  DISP file:        delivery_data_cal_disp/input_data/disp/OPERA_L3_DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v1.0_20251027T005420Z.nc
  Frame ID:         8882
  UNR lookup:       delivery_data_cal_disp/input_data/unr/grid_latlon_lookup_v0.2.txt
  UNR grid dir:     delivery_data_cal_disp/input_data/unr

  UNR version:         0.2
  UNR type:         variable
Worker Settings:
  Workers:          4
  Threads/worker:   1
  Total threads:    4
  Block shape:      (512, 512)

Dynamic Ancillary Files:
  algorithm_parameters_file: delivery_data_cal_disp/configs/algorithm_parameters.yaml
  los_file: delivery_data_cal_disp/input_data/static_input/OPERA_L3_DISP-S1-STATIC_F08882_20140403_S1A_v1.0_line_of_sight_enu.tif
  dem_file: delivery_data_cal_disp/input_data/static_input/OPERA_L3_DISP-S1-STATIC_F08882_20140403_S1A_v1.0_dem.tif

Workflow Status: READY
======================================================================

When execution is finished, a directory output/ should be created in delivery_data_cal_disp/ containing a single pair of .nc and .png files:

$ ll delivery_data_cal_disp/
total 12
-rw-r--r-- 1 rileykk cloud-user  529 Jan 22 17:49 cal_disp.log
drwxr-xr-x 2 rileykk cloud-user   61 Jan 21 01:29 configs
drwxr-xr-x 2 rileykk cloud-user 4096 Jan 21 01:36 golden_output
drwxr-xr-x 6 rileykk cloud-user   62 Jan 21 00:45 input_data
drwxr-xr-x 2 rileykk cloud-user 4096 Jan 22 17:49 output

$ ll delivery_data_cal_disp/output/
total 865076
-rw-r--r-- 1 rileykk cloud-user 878477162 Jan 22 17:49 OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260122T174924Z.nc
-rw-r--r-- 1 rileykk cloud-user   7359504 Jan 22 17:49 OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260122T174924Z.png

Running the Quality Assurance test

Now that we've successfully executed the SAS container and generated outputs, the last step is to perform a QA check against the expected outputs.

A Python program to compare CAL-DISP products generated by CAL-DISP-SAS with the expected “golden dataset” output is included in the Docker image. The cal-disp application can run a "validate" workflow that accepts two input files: the golden dataset and the test dataset.

The docker command to run this is:

docker run --rm --user $(id -u):$(id -g) \
    --volume $(pwd):/home/work \
    --workdir /home/work \
    opera/cal-disp:interface_0.1 cal-disp validate \
    delivery_data_cal_disp/golden_output/<CAL-DISP filename> delivery_data_cal_disp/output/<CAL-DISP filename>

For example, if the SAS was run using the example command above and the result is in the output/ directory, the validation program can be run as as follows:

docker run --rm --user $(id -u):$(id -g) \
    --volume $(pwd):/home/work \
    --workdir /home/work \
    opera/cal-disp:interface_0.1 cal-disp validate \
    delivery_data_cal_disp/golden_output/OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260121T010950Z.nc \
    delivery_data_cal_disp/output/OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260122T174924Z.nc

This test is currently passing on an AMD EC2 instance (r6a.4xlarge).

Small-size test case sample validation output:

2026-01-22 17:51:54 - cal_disp.validate - INFO - Comparing OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260122T174924Z.nc against OPERA_L4_CAL-DISP-S1_IW_F08882_VV_20220111T002651Z_20220722T002657Z_v0.1_20260121T010950Z.nc
2026-01-22 17:51:54 - cal_disp.validate - INFO - Validating product structure...
2026-01-22 17:51:54 - cal_disp.validate - INFO -   ✓ calibration dimensions
2026-01-22 17:51:54 - cal_disp.validate - INFO -   ✓ source_calibration_file_list is scalar string
2026-01-22 17:51:54 - cal_disp.validate - INFO -   ✓ source_data_file_list is scalar string
2026-01-22 17:51:54 - cal_disp.validate - INFO -   ✓ source_data_satellite_names is scalar string
2026-01-22 17:51:54 - cal_disp.validate - INFO - Validating main group...
2026-01-22 17:51:55 - cal_disp.validate - INFO -   ✓ calibration_std
2026-01-22 17:51:55 - cal_disp.validate - INFO -   ✓ calibration
2026-01-22 17:51:55 - cal_disp.validate - INFO - Validating auxiliary group...
2026-01-22 17:51:56 - cal_disp.validate - INFO -   ✓ north_south
2026-01-22 17:51:56 - cal_disp.validate - INFO -   ✓ east_west
2026-01-22 17:51:56 - cal_disp.validate - INFO -   ✓ up_down
2026-01-22 17:51:56 - cal_disp.validate - INFO - ✓ Validation passed

Clone this wiki locally