Parameter Definition and Updating

GUIDE FOR MODULE LEADS
Walk Through Video HERE

Project Purpose: To support the continued development and improvement of model parameterization.

Goals

• Goal A: Encourage developers from other countries or contexts to carefully consider which parameter values to assume, rather than adopting defaults uncritically.
  • Local variables send a stronger signal to developers to revisit, universal may be more broadly applied.
• Goal B: Provide clear guidance for future developers on which parameters can be used for calibration.
  • Parameters that are universal and local may be treated as more “fixed.”
  • Parameters that are undetermined and leveraged for calibration can be adjusted more flexibly to achieve tighter calibration in future iterations
• Goal C: Define prior ranges to reflect uncertainty in results

Parameter Classifications

• Undetermined
  • Structural or Design Decision: Parameters defined by developers that are inherent to the structure of the code. The comment in the prior_note column is either “design decision” (eg. polling frequency, date offset) or “structural” (e.g. age group thresholds). These two comments can be interchangeable.
  • Calibration: Parameter that has been clearly adjusted for calibration as defined in write-up or code documentation (e.g. death risk in measles was indicated to be calibrated). The comment in the prior_note column is set to “calibration”.
• Local
  • Parameters that are location-specific and may change country-to-country (e.g. probability of referral, transmission rates, age of treatment, seasonality, relative prevalence). It can also be more subtle in nature and have a local-element that directs the parameters to be local (e.g. relative rate of complication in second parity may be related to the quality of care received in the previous deliveries). The source is often based in Malawi or LIC settings. Parameters must be revisited by future developers.
  • Note: It is OK for the reference of these parameters to be universal, what is more important is the use-case and intention behind it
• Universal
  • Universally fixed regardless of the context in which it it assumed (eg. test specificity, vaccine efficacy, probability severe case of disease, probability of death on treatment, timing of death, disease resolution, symptom onset, age of disease onset, relative rate, relative incidence). Also recommended to be reviewed by future developers.
  • Note: It is OK for the reference of these parameters to be local, what is more important is the use-case and intention behind it
• Scenario:
  • Parameter used to define a future hypothetical scenario (e.g. ART coverage from 2026 onwards.)
• Calibration data
  • Parameters that are not necessarily ‘parameters’ but rather data used to calibrate the model

Format of parameter file

• Columns: parameter_name, value, param_label, prior_min, prior_max, prior_note, resource

Column	Content
parameter_name	Lowercase, separated by underscore
value	Number or list or text
param_label	Undetermined, scenario, universal, local, calibration data
prior_min	Number or list
prior_max	Number or list
prior_note	Vague prior, structural assumption, design decision, calibration, 95% confidence interval
resource	Free text (preference for link to source, otherwise, if not citation OK). At this time most have been left blank. This does not indicate that the source is missing, but rather that it has not been documented.

Labeling Strategy:

• When reviewing parameters, label it based on how each parameter was handled in the development process. If uncertain, label it using the approach that would be best for future applications.
• When unsure whether a parameter is local or universal, err on the side of labeling it as local. This provides a stronger signal for future developers to revisit and assess whether localization is warranted.
• Any parameters where no clear reference is provided and the write-up does not explicitly state whether they were calibrated or not, we default to assume that the developer has obtained the value through "triangulation". We set this value as either local/universal.
• Any parameter that has been labeled as ‘dummy’ or ‘assumed’ but is not explicitly calibrated is believed to be a ‘placeholder’ value. These can be labeled as local/universal

Priors

• Broad priors are indicators of uncertainty. Oftentimes, this will be associated with a universal or local parameter of which the source has not been confirmed. This serves as a flag for future developers to review.

References

• Overview: References were not thoroughly documented in this process. References are most clearly outlined in the module write-ups. At this time, a reference is only documented if it was previously noted in the CSV file, and it is left blank otherwise. This does not mean that it is missing.

Multiple Parameter Files

• All parameters across all files are to be documented in the format documented above (when applicable).
• Clean-up:
  • If files are not referenced anywhere in the codebase, OK to delete. All other files (even if only used for analysis) can be kept in the folder
  • The structuring of the parameter files (e.g. all stored under ResourceFile_modulename folder) is to be done by software engineering team

Hardcoding

• OK to remain hardcoded:
  • What is true by definition (e.g. 7d in the week, DALY codes, Under 5 OPD age <5, …)
  • Vestigial features (e.g. priority of a service)
  • Appointment footprints
  • Age ranges in linear models
  • Anything related to logging, tests, or checks
• NOT OK to be hardcoded:
  • Date timings and delay windows (intrinsically linked to transmission and death rates)

Recommended vague priors

The following are appropriate ranges if no information is available to ‘tighten’ the priors

• Odds ratio seeking care: 0.01 – 100
• Beta transmission baseline (sin): 0 – 1
• Beta transmission scale (sin): 0 – 5
• Phase (annual): fixed at 12
• Phase shift (annual): 0 – 11
• Vaccine efficacy: 0 – 1
• Relative risk (RR): 0.01 – 100
• Post-treatment check intervals (months): 1 – 24 months
• Minimum age seek care (if not a child): 0 – 20 age
• Polling interval: 1 – 12 months

Practical Revision Recommendations

• PARAMETER initialization in module:
  • Typing: Ensure proper typing of INT v. REAL
  • Nomenclature and definitions: Ensure nomenclature and definitions align with module use-case
  • Order: Change the ordering of parameters to group them into categories; Ensure this order is reflected in the CSV file.
• Parameter in CSV files:
  • Values: Ensure that all parameter values are the same as the original hard-coded values
  • Specific categories:
    • Age groups: Ensure that age groups are all marked as ‘undetermined’ and ‘structural’
  • Priors
    • Less Vague: Update priors to be ‘less vague’ based on developer input, especially those that do have references with ranges that can be based on confidence intervals
  • Addt. Note
    • Review: Any time there were comments in the CSV file, I have kept them in an additional column. Please review which notes are helpful to keep or remove so that we can work towards having a clean, structured parameter CSV file

Future Recommendations (Not required to be completed in 2025 revisions)

• References
  If no reference is required for a given parameter (undetermined classification), fill the “resource” column with “N/A”.
  • Otherwise (for local and universal variables)
    • If reference is available in write-up, include it in the CSV file
      • Update priors based on range in reference (if applicable)
    • If reference is required but not found:
      • Fill the resource column with “Missing” and set prior min-max to the widest possible. It will be the responsibility of module developers to fill missing info.
  • If reference at the end of the paragraph and no other reference cited, assume that it is the reference for the information contained in the paragraph
• Documentation
  • Consistency across write-up and module
  • All new parameters added to the write-up