Normalization step failure-collection/review ideas

[colleenXu]

[EDITED after discussion with Evan 2025-09-18]

Background

I've previously brought up the complications of the normalization step from my POV, based on my experience doing it in my custom notebooks. I collect data on failures (broken down by type of failure) and review/analyze those failures and other normalization stuff. I've been asked to share some details on what I'm doing, so that's what this issue is for.

While doing all this is more work, I've found it very helpful for:

• finding NodeNorm issues/shortcomings
• finding errors in the resource itself (that developers appreciated being informed of specific errors and fixed!)
• figuring out what to use as the original ID when the resource contains multiple options

Details

I catch these types of NodeNorm mapping failures:

• NodeNorm returned None - save the input ID. ORION's current NodeNorm failure file does this
• NodeNorm clique is the wrong primary category - save the input ID and that NodeNorm category. This requires as input an expected category/list of expected categories (which can be tricky to include)
• NodeNorm clique's doesn't have a primary label - save the input ID. Evan agreed to add this. I've heard that the UI doesn't like it when Nodes don't have a human-readable label so this
• Unexpected errors (caught with try-except) - save the input ID and that NodeNorm response

Then I save and print summary statistics for each failure type: how many input IDs affected and how many rows removed as a result.

I've done these kinds of reviews/analyses:

• A resource provides two ID columns for an entity - compare the NodeNorm mappings from each column for differences.
  • If they differ, manually compare the input IDs, NodeNorm mappings, and original concept (string) to find which column of input IDs best captures the original concept
• Compare the names provided by the resource to the NodeNorm primary labels
• For 1 ID column from the resource, compare the input IDs, NodeNorm mappings, and original concept (string) to see if the original concept is being accurately represented by the input ID/NodeNorm mapping.

[colleenXu]

Notes from Evan and my discussion 2025-09-18:

• merging Nodes is necessary (cannot have two with the same ID), vs merging Edges is "optional"
• "default" merging Nodes is based on the same normalized primary ID and involves trying to reconcile different Node properties
• "default" merging Edges could be based on "all single-value properties (not lists/multivalued) are the same". At minimum, the properties that should be the same are:
  • subject (normalized primary ID)
  • predicate
  • qualifier-set
  • object (normalized primary ID)
  • KL
  • AT
• Then to actually merge these "duplicate" edges, it's easy to concatenate the properties that are lists (ex: publications, source_record_ids). This may involve edits to the biolink-model to explicitly make some properties single-value or multi-valued, like Make original subject/original object plural, multivalued biolink/biolink-model#1574

See biolink/biolink-model#1574 (comment) for examples of resource data that will lead to merging issues. Evan and I decided that "scenario 3" (all single-value properties are the same) would be the only scenario where the edges made from the rows would be merged.