Skip to content

[1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release #1539

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: dev
Choose a base branch
from

Conversation

EmilyDeardorff
Copy link
Contributor

@EmilyDeardorff EmilyDeardorff commented May 23, 2025

Updates the CatFIM site comparison tool to make the outputs better suited to be loaded into HydroVis.

Resolves Issue #1517

Changes

  • /tools/catfim/catfim_sites_compare.py
    • Replacing the status-specific column names with old_version_status and current_version_status and then add two columns, old_version and current_version, that have the HAND version ID (i.e. 4.6.1.4)
    • Saving the gained and lost coverage data as a CSV (in addition to the GPKG files that are currently being output)
    • Update the filename of the output sites CSVs and GPKGs so they have “_sites” in the file name.
    • Added a debug mode that allows the developer to only run through 100 unique lid/magnitude combinations (to shorten testing time).
    • Calculate the percent change for each lid/magnitude combination and add it to the output files.
      • added_area_percent is the percentage of inundated area in the newer CatFIM version that was not inundated in the previous version. It is calculated for each site and magnitude as follows: (area inundated only in new CatFIM version / total area inundated in new CatFIM version) * 100
      • removed_area_percent describes the percentage of the previous inundated area that is no longer being inundated in the newer CatFIM version. It is calculated for each site and magnitude as follows: (area inundated only in old CatFIM version / total area inundated in old CatFIM version) * 100

Testing

Ran for two most recent versions of stage- and flow-based CatFIM and the updates worked as expected. Run for most recent two versions - stage-based. Tested both with and without -g flag, which toggles whether the geopackages get produced.

Deployment Plan (For developer use)

How does the changes affect the product?

  • Code only?
  • If applicable, has a deployment plan be created with the deployment person/team?
  • Require new or adjusted data inputs? Does it have start, end and duration code (in UTC)?
  • If new or updated data sets, has the FIM code been updated and tested with the new/adjusted data (subset is fine, but must be a subset of the new data)?
  • Require new pre-clip set?
  • Has new or updated python packages?

Issuer Checklist (For developer use)

You may update this checklist before and/or after creating the PR. If you're unsure about any of them, please ask, we're here to help! These items are what we are going to look for before merging your code.

  • Informative and human-readable title, using the format: [_pt] PR: <description>
  • Links are provided if this PR resolves an issue, or depends on another other PR
  • If submitting a PR to the dev branch (the default branch), you have a descriptive Feature Branch name using the format: dev-<description-of-change> (e.g. dev-revise-levee-masking)
  • Changes are limited to a single goal (no scope creep)
  • The feature branch you're submitting as a PR is up to date (merged) with the latest dev branch
  • pre-commit hooks were run locally
  • Any change in functionality is tested
  • New functions are documented (with a description, list of inputs, and expected output)
  • Placeholder code is flagged / future todos are captured in comments
  • CHANGELOG updated with template version number, e.g. 4.x.x.x
  • Add yourself as an assignee in the PR as well as the FIM Technical Lead

Merge Checklist (For Technical Lead use only)

  • Update CHANGELOG with latest version number and merge date
  • Update the Citation.cff file to reflect the latest version number in the CHANGELOG
  • If applicable, update README with major alterations

@EmilyDeardorff EmilyDeardorff self-assigned this May 23, 2025
@EmilyDeardorff EmilyDeardorff added the CatFIM NWS Flood Categorical HAND FIM label May 23, 2025
@EmilyDeardorff EmilyDeardorff changed the title WIP [1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release [1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release May 23, 2025
@EmilyDeardorff EmilyDeardorff requested review from RobHanna-NOAA and removed request for RobHanna-NOAA May 23, 2025 21:47
@EmilyDeardorff EmilyDeardorff changed the title [1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release WIP [1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release May 23, 2025
@EmilyDeardorff EmilyDeardorff changed the title WIP [1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release [1pt] Update CatFIM site comparison tool to have the proper outputs for HydroVis release Jun 11, 2025
@EmilyDeardorff EmilyDeardorff marked this pull request as ready for review June 11, 2025 21:25
RobHanna-NOAA
RobHanna-NOAA previously approved these changes Jun 18, 2025
Copy link
Contributor

@RobHanna-NOAA RobHanna-NOAA left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested against 4.5.11.1 and 4.6.1.4 for both stage based and flow based and looks good

RobHanna-NOAA
RobHanna-NOAA previously approved these changes Jun 30, 2025
@RobHanna-NOAA
Copy link
Contributor

Ran full scale Flow Base CatFIM for 4.8.6.1 and the comparison tool again it and 4.6.1.4. Both look great. Did some analysis on the sites and the libraries against the datasets for sites / libraries against 4.6.1.4 and they looked good. Did some analysis of the compare tool results against the two and it looks great as well.. Wahoo!!! Flow based is perfect.

Now, we just have to finish getting stage based happy.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CatFIM NWS Flood Categorical HAND FIM
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants