Enhanced design of Study and StudyResult models

[RichardBruskiewich]

We wish to improve the representation of supporting scientific studies in the Biolink Model.

Working group that this request originate from?

The need for representing this type of data originated from the activity to design KGX pass through ingest of (RENCI) ICEES data from Phase 2 Translator, which represented supporting study data for edges as a block of attributes.

Describe the solution you'd like

The current Biolink Model (release 4.3.2) defines a slot has_supporting_studies which is generally bound to the Association class. This slot has a range of biolink:Study but is not defined as inlined in Association, so only the study connections are only documented using a list of Study identifiers, which may or may not be embedded within the knowledge graph.

The Phase 2 ICEES dataset has Association (child class) instances with a list of result data documented as a list of TRAPI-style attributes. We would rather wrap this results as proper instances of StudyResult. These data entries should be properly connected to their source Study and associated knowledge Association.

Additional information to support this request (optional)

An example of such KGX (jsonl) 'edges' using attributes for study results for ICEES is at
https://stars.renci.org/var/plater/bl-4.2.1/icees-kg/8-20-2024/edges.jsonl.gz

Tag relevant members for discussion
@sierra-moxon @mbrush @gglusman

[RichardBruskiewich]

Is the relationship between edges and their supporting study data, structured like:

Association -> Study -> StudyResult

where an Association simply points to the Study but somehow is also related to results in that study.

Or rather, like

Association -> StudyResult <- Study

This latter treating StudyResult like a kind of relational database-like join table, thus having a composite identifying content pointing both to the Association and to the Study with which it is related. This could but, doesn't necessarily require, that a StudyResult be a separate node in a knowledge graph, but it does point to the necessity of proper data cross-referencing.

[RichardBruskiewich]

As mentioned above, the Integrated Clinical and Environmental Exposures Service (ICEES) KGX dataset has edges with supporting study results documented as TRAPI-style attributes, somewhat as follows, where study results are tagged with icees_cohort_identifier type of attributes, with embedded list of attributes containing study statistics for the particular asserted association statement (the following json is just formatted with extra newlines for readability.. the original string is all on one line) :

{
    "subject": "PUBCHEM.COMPOUND:2083",
    "predicate": "biolink:positively_correlated_with",
    "object": "MONDO:0007079",
    "primary_knowledge_source": "infores:icees-kg",
    "attributes": [
                       "{
    \"attribute_type_id\": \"icees_cohort_identifier\",
    \"value\": \"PCD_UNC_patient_2020_v6_binned_deidentified|pcd|v6|2024_03_20_21_18_22\",
    \"attributes\": [
        {
            \"attribute_type_id\": \"chi_squared_statistic\",
            \"value\": 26.38523077566414
        },
        {
            \"attribute_type_id\": \"chi_squared_dof\",
            \"value\": 1
        },
        {
            \"attribute_type_id\": \"chi_squared_p\",
            \"value\": 2.7967079822744063e-07
        },
        {
            \"attribute_type_id\": \"total_sample_size\",
            \"value\": 4753.0
        },
        {
            \"attribute_type_id\": \"fisher_exact_odds_ratio\",
            \"value\": 3.226188583240579
        },
        {
            \"attribute_type_id\": \"fisher_exact_p\",
            \"value\": 2.8581244515361156e-06
        },
        {
            \"attribute_type_id\": \"log_odds_ratio\",
            \"value\": 1.1713014352915974
        },
        {
            \"attribute_type_id\": \"log_odds_ratio_95_ci\",
            \"value\": [
                0.6996904681742875,
                1.6429124024089072
            ]
        }
    ]
}

[RichardBruskiewich]

To better capture this in the Biolink Model, we could consider adding a new slot defined as follows:

  has supporting study results:
    range: study result
    multivalued: true
    is_a: association slot
    description: >-
      Connects an association to instances of supporting study result

This slot would be added to the biolink:Association with helpful slot_usage directive (other association metadata not shown here)

association:
   slots:
     - has supporting study results   
   slot_usage:
      has supporting study results:
        inlined_as_list: true

This slot would basically store the data as an inlined list of full StudyResult instances.

This slot not (likely) replace the existing biolink:has_supporting_studies slot which could still be used to annotate some edges with studies which support them but for which, the specific statistics of the support are not (yet) annotated on the Association.

However, each StudyResult would necessarily need some kind of internal reference back to the specific study to which it belongs. Unless an Association only has one supporting Study (however many associated study results it has), the biolink:has_supporting_studies slot values would somehow need to be mapped back to their specific study results (stored with the Association).

One possible alternative to embedding a StudyResult data within an Association as inlined data, is to record it as a separate graph node with a globally unique identifier, then simply store the identifier (as a string) in a has supporting study results slot list of such identifiers.

Such StudyResult objects identifiers could be listed under a given Study using a multivalued slot perhaps simply called has study result?

[RichardBruskiewich]

The last consideration in this model is how best to relate a StudyResult back to its source Study.

If there is no general need to navigate from randomly selected StudyResult instances back to their source Study, then the aforementioned has study result aggregation of StudyResult likely suffices.

However, if in the ICEES use case, with the one-to-one relationships of Association to StudyResult and StudyResult to Study, with inlined aggregation of StudyResult instances on either side (embedded in the Association or in the Study knowledge subgraph), it seems necessary to store some extra cross-referencing identifier in the StudyResult itself.

If the StudyResult is inlined under the has supporting study results slot within the Association, then it needs a slot (perhaps named 'from study') to record the identifier of the Study from which it originated. Conversely, if the StudyResult is inlined under the has study results slot (perhaps named 'from association'?) within the Study, then it should have a slot to record some global identifier of the related Association.

[RichardBruskiewich]

One more design question relates to the existing Association biolink:has_supporting_studies slot with range Study: should this slot also be inlined_as_list?

A basic argument against this idea is that a given Study is actually usually a very large dataset. Moreover, a given Association may only be supported by one (or a very few) study results from the Study.

[sierra-moxon]

After talking with Richard for a long time, I am of the opinion that this is the model we should consider:

association:
  has_supporting_studies (range: Study, multivalued)
study:
   has supporting study results (range: StudyResult, multivalued)

The inlining discussion is separate, but you could have both of these "has" attributes inlined in the JSONlines output.

[RichardBruskiewich]

Given a couple of query use cases:

1. Given an association, retrieve scores from studies supporting the knowledge assertion
2. Given a specific study, return all its study results (possibly filtered)

Assuming that every association has a specific score values, mapped one-to-one, derived from supporting studies with.

At least two data models may be considered:

Data Model option 1: with StudyResult entries only aggregated under their source Study.

If a StudyResult (not just the id values of instances of Study alone) generally support many distinct Association edges in the knowledge graph (as recorded as a list of globally unique association_id values in each StudyResult), then this model may be the preferred (a DRY StudyResult) approach.

a = Association(id="given_association_id")

# The has_supporting_studies slot is a list[str] of study_id's
for study_id in a.has_supporting_studies:

# Retrieve the study instance in a second query...
 study = Study(id==study_id)

    # ... then iterate through the list of its StudyResult
    for sr in study.has_supporting_study_results:

        # We have to assume that the StudyResult has a slot 'association_ids' that 
        # maintains a the list[str] of identifiers pointing to possibly multiple Association's it supports

        if a.id in sr.associations_ids:   
            score = sr.score. 
            break

returns a study_id, score for a given Association. This data model is DRY with respect to StudyResult instances and supports swift retrieval and filtering of a given Study:

target_study = Study(id=="target_study_id")

def my_filter(sr: StudyResult) ->bool:
    # filtering code returning 'True' if the result is of interest

results_of_interest = [sr for sr in target_study.has_supporting_study_results if my_filter(sr)]

If StudyResult (not just Study) instances support many distinct instances of Association in the knowledge graph, then this data model may make sense.

Data Model option 2: with StudyResult entries directly embedded inside the Association they support.

Alternately, if every StudyResult of every Study, all (or mostly?) maps one-to-one onto one specific Association (i.e. like ICEES data) then:

Practically speaking, this StudyResult data is recorded as inlined_as_list,
but also, the StudyResult needs to includes a slot value pointing back to its source Study
e.g. as found within the existing Phase 2 ICEES KGX data (modelled as embedded TRAPI-like 'attributes')

a = Association(id="given_association_id")

for sr in a.has_supporting_study_results:  
	
    study_id = sr.from_study
    score = sr.score

also returns a study_id, score for a given Association.

Although aggregating StudyResult instances ('nodes') directly under their Study instances (as slot values) seems intuitively correct, if most such StudyResult instances mainly support one single Association, and are data mostly of value to users locally accessing a specific Association, then Option 2 remains DRY and also, retrieval efficient.

However, retrieval with filtering of all StudyResult values from a given Study known to a knowledge graph becomes a little more challenging (i.e. it is easier to directly do that task using the Option 1 data model), since such StudyResult instances are distributed in Association instances throughout the knowledge graph, rather than under a given Study.

def my_filter(sr: StudyResult) ->bool:
    # filtering code returning 'True' if the result is of interest

results_of_interest: list[StudyResult] = []

for a in associations:
    for sr in a.has_supporting_study_results: 
        if sr.from_study =="target_study_id" and my_filter(sr):
            results_of_interest.append(sr)