Integrating semantic annotation tasks directly into common tools

[edan-bainglass]

Problem

Even with benefits and standards of semantic annotation made clear, the task is time consuming.

Proposed solution

Integrate the tasks directly into common research tools to automate as much as possible.

---

@jan-janssen Semantikon was brought to my attention. I am approaching this differently here, but with a similar goal in mind. The README there is not great - could use some work. Maybe we discuss on a call in the coming weeks if you have time.

[jan-janssen]

@edan-bainglass I agree both approaches as similar, but @samwaseda can maybe explain more. The main focus for the design of semantikon was the option to easily annotate python functions and connect them to an ontology. To me it is not yet clear how you use your pydantic data classes in different workflow frameworks. Can I think about them as the data objects transferred between two nodes of a workflow graph? Or are these more general annotations for the individual nodes? Or both? Still I think this is exactly the missing documentation you mentioned.

[edan-bainglass]

@jan-janssen hmm, I'd say both... soon. At the moment, the implementation is limited to the schemas for the AiiDA common workflows. However, the upcoming AiiDA release will include pydantic models for all ORM models. I plan to integrate my semantic models into these ORM pydantic models, so that all AiiDA Data nodes will carry semantic meaning out of the box.

[samwaseda]

I took a quick look at the example notebook. It looks like common workflow schemas mainly deals with the data sets, which can be naturally integrated into semantikon. In particular, if there's a function like this:

def parse_output(file_or_object_to_parse):
    ...
    return RelaxOutputs(forces=forces, total_energy=total_energy)

This code above can already be used in semantikon out of the box, or alternatively you can also write:

from semantikon.metadata import u

def parse_output(file_or_object_to_parse) -> tuple[
    u(float, description="The forces on the atoms.", iri=f"{BASE_PREFIX}/Forces", units="eV/Å"),
    u(float, description="The total energy...", iri=f"{BASE_PREFIX}/scf/TotalEnergy", units="eV")
]:
    ...
    return forces, total_energy

BUT more importantly, semantikon deals with the knowledge graph of a workflow, meaning if multiple components are connected, the resulting data flow can be traced back. In this example (sorry quite outdated and probably also difficult to understand), I show how to ontologically extract the information about the vacancy formation energy, which goes beyond dataset information such as total energy, because it involves multiple steps.

[PeterKraus]

I would argue it's perhaps a lower hanging fruit to annotate at the dataset level than at the workflow level. At this stage, we shouldn't discourage the former in favour of the latter, in my view.

I have recently found https://h5rdmtoolbox.readthedocs.io/en/latest/ which seems to be implementing the "missing link" between user-friendly HDF5 via xarray and proper semantic annotations.