|
| 1 | +# Benchmark Schema |
| 2 | + |
| 3 | +The OpenMC Fusion Benchmarks (OFB) project uses a modular, schema-driven format to ensure that all benchmark `specifications` follow a consistent, machine-validated structure. |
| 4 | + |
| 5 | +## Validation |
| 6 | + |
| 7 | +Validation can be performed anually using the OFB Python API: |
| 8 | + |
| 9 | +```python |
| 10 | +import openmc_fusion_benchmarks as ofb |
| 11 | + |
| 12 | +ofb.validate_benchmark('benchmark_name') |
| 13 | +``` |
| 14 | + |
| 15 | +Alternatively, you can run the `validate_all_benchmarks.py` script located in the repository’s `scripts/` folder. This script iterates through all benchmark `specifications.yaml` files and validates them against the repository’s `benchmark_schema`. |
| 16 | + |
| 17 | +Validation output looks like: |
| 18 | + |
| 19 | +```shell |
| 20 | +🔍 Validating benchmark file: benchmark_name |
| 21 | +✅ benchmark_name is valid! |
| 22 | +``` |
| 23 | + |
| 24 | +Or in case of validation errors: |
| 25 | + |
| 26 | +```shell |
| 27 | +🔍 Validating benchmark file: oktavian_al |
| 28 | +❌ 1 errors found in oktavian_al: |
| 29 | + - None is not of type 'object' (at ['materials', 1, 'composition', 'data']) |
| 30 | +``` |
| 31 | + |
| 32 | +In the example above, the material with `id` 1 is missing a defined composition. |
| 33 | + |
| 34 | +Validation is also automatically triggered when instantiating a `benchmark` object: |
| 35 | + |
| 36 | +```python |
| 37 | +import openmc_fusion_benchmarks as ofb |
| 38 | + |
| 39 | +benchmark = ofb.OpenmcBenchmark(name='benchmark_name') |
| 40 | +``` |
| 41 | +```shell |
| 42 | +🔍 Validating benchmark file: benchmark_name |
| 43 | +✅ benchmark_name is valid! |
| 44 | +``` |
| 45 | + |
| 46 | +## Purpose of the Schema |
| 47 | + |
| 48 | +To guarantee interoperability and automation, each `specifications.yaml` file must conform to a predefined [JSON Schema](https://json-schema.org/). This enables: |
| 49 | + |
| 50 | +- **Automatic validation** of input files |
| 51 | +- **Improved error reporting** for malformed benchmarks |
| 52 | +- **Robust tooling** for model generation and testing |
| 53 | +- **Future extensibility** of the specification format |
| 54 | + |
| 55 | +## Schema-to-Specifications Mapping |
| 56 | + |
| 57 | +The `benchmark_schema` outlines the main _sections_ expected in a `specifications.yaml` file: |
| 58 | + |
| 59 | +```yaml |
| 60 | +$ref: "#/components/schemas/Benchmark" |
| 61 | +components: |
| 62 | + schemas: |
| 63 | + Benchmark: |
| 64 | + type: object |
| 65 | + required: |
| 66 | + - metadata |
| 67 | + - materials |
| 68 | + - geometry |
| 69 | + - sources |
| 70 | + - settings |
| 71 | + - tallies |
| 72 | + properties: |
| 73 | + metadata: |
| 74 | + $ref: '#/components/schemas/Metadata' |
| 75 | + materials: |
| 76 | + type: array |
| 77 | + items: |
| 78 | + $ref: '#/components/schemas/Material' |
| 79 | + ... |
| 80 | +``` |
| 81 | + |
| 82 | +Some sections are optional and included only when relevant to the benchmark: |
| 83 | + |
| 84 | +```yaml |
| 85 | + ... |
| 86 | + irradiation_schedule: |
| 87 | + $ref: '#/components/schemas/IrradiationSchedule' |
| 88 | + uncertainty_quantification: |
| 89 | + $ref: '#/components/schemas/UncertaintyQuantification' |
| 90 | +``` |
| 91 | +
|
| 92 | +Beyond defining the structure at a high level, the `schema` specifies the internal _hierarchy_ and _data types_ for each object: |
| 93 | + |
| 94 | +```yaml |
| 95 | +Tally: |
| 96 | + type: object |
| 97 | + required: [name, particle, filters, scores] |
| 98 | + properties: |
| 99 | + name: |
| 100 | + type: string |
| 101 | + ... |
| 102 | + particle: |
| 103 | + type: string |
| 104 | + enum: [neutron, photon, electron, positron] |
| 105 | + filters: |
| 106 | + type: array |
| 107 | + items: |
| 108 | + type: object |
| 109 | + required: [type, values] |
| 110 | + ... |
| 111 | +``` |
| 112 | + |
| 113 | +Each section in a benchmark’s `specifications.yaml` is _validated_ against its corresponding `schema` definition. |
| 114 | + |
| 115 | +For example, a `density` object in the `specifications` might appear as: |
| 116 | +```yaml |
| 117 | +density: |
| 118 | + value: 0.997 |
| 119 | + units: g/cm3 |
| 120 | +``` |
| 121 | + |
| 122 | +Its `schema` counterpart ensures the expected structure and value types: |
| 123 | + |
| 124 | +```yaml |
| 125 | +density: |
| 126 | + type: object |
| 127 | + properties: |
| 128 | + value: |
| 129 | + type: number |
| 130 | + units: |
| 131 | + type: string |
| 132 | + enum: [g/cm3] |
| 133 | + required: [value, units] |
| 134 | +``` |
| 135 | + |
| 136 | +Likewise, a complete `material` entry in the `materials` list could be: |
| 137 | + |
| 138 | +```yaml |
| 139 | +- id: 1 |
| 140 | + name: Water |
| 141 | + composition: |
| 142 | + composition_type: element |
| 143 | + fraction_type: atomic |
| 144 | + data: |
| 145 | + H: 0.67 |
| 146 | + O: 0.33 |
| 147 | + density: |
| 148 | + value: 0.997 |
| 149 | + units: g/cm3 |
| 150 | +``` |
| 151 | + |
| 152 | +And its definition in the `schema` would be: |
| 153 | + |
| 154 | +```yaml |
| 155 | +Material: |
| 156 | + type: object |
| 157 | + required: [id, name, composition, density] |
| 158 | + properties: |
| 159 | + id: |
| 160 | + type: integer |
| 161 | + name: |
| 162 | + type: string |
| 163 | + composition: |
| 164 | + type: object |
| 165 | + properties: |
| 166 | + composition_type: |
| 167 | + type: string |
| 168 | + fraction_type: |
| 169 | + type: string |
| 170 | + enum: [atomic, weight] |
| 171 | + data: |
| 172 | + type: object |
| 173 | + additionalProperties: |
| 174 | + type: number |
| 175 | + required: [composition_type, fraction_type, data] |
| 176 | + density: |
| 177 | + ... |
| 178 | +``` |
| 179 | + |
| 180 | +## Unified Schema Format |
| 181 | + |
| 182 | +The OFB schema is a **single, unified JSON Schema file** that defines the complete structure of a valid `specifications.yaml` file. It includes all top-level sections—such as `metadata`, `geometry`, `materials`, `source`, `tallies`, and others—and their corresponding nested fields. |
| 183 | + |
| 184 | +Although the schema is written modularly (with subschemas for each section), it is maintained as **one file** for simplicity, validation consistency, and ease of distribution. |
0 commit comments