Skip to content

Commit 70d3d23

Browse files
Copilotyarikoptic
andcommitted
Add validate_bids-examples workflow for testing migrations
- Add workflow based on PR #1775 to test migrations on example datasets - Uses bst migrate command to migrate datasets from bids-examples - Runs validation before and after migration with deno validator - Skips check-only migrations (inheritance overloading and TSV prefix checks) - Adapted to use new 'bst migrate all' command instead of 'bst migrate-datasets' Co-authored-by: yarikoptic <[email protected]>
1 parent ca2f71e commit 70d3d23

File tree

1 file changed

+111
-0
lines changed

1 file changed

+111
-0
lines changed
Lines changed: 111 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,111 @@
1+
name: validate_datasets
2+
3+
on:
4+
push:
5+
branches: ["master"]
6+
pull_request:
7+
branches: ["**"]
8+
9+
concurrency:
10+
group: ${{ github.workflow }}-${{ github.ref }}
11+
cancel-in-progress: true
12+
13+
jobs:
14+
build:
15+
strategy:
16+
fail-fast: false
17+
matrix:
18+
platform: [ubuntu-latest]
19+
bids-validator: [master-deno]
20+
python-version: ["3.11"]
21+
22+
runs-on: ${{ matrix.platform }}
23+
24+
env:
25+
TZ: Europe/Berlin
26+
FORCE_COLOR: 1
27+
28+
steps:
29+
- uses: actions/checkout@v4
30+
31+
# Setup Python with bst
32+
- uses: actions/setup-python@v5
33+
with:
34+
python-version: ${{ matrix.python-version }}
35+
- name: "Install build dependencies"
36+
run: pip install --upgrade build twine
37+
- name: "Build source distribution and wheel"
38+
run: python -m build tools/schemacode
39+
- name: "Check distribution metadata"
40+
run: twine check tools/schemacode/dist/*
41+
- name: "Install bst tools from the build"
42+
run: pip install $( ls tools/schemacode/dist/*.whl )[all]
43+
- name: "Produce dump of the schema as schema.json"
44+
run: bst -v export --output src/schema.json
45+
46+
- uses: denoland/setup-deno@v1
47+
if: "matrix.bids-validator == 'master-deno'"
48+
with:
49+
deno-version: v1.x
50+
51+
- name: Install BIDS validator (master deno build)
52+
if: "matrix.bids-validator == 'master-deno'"
53+
run: |
54+
pushd ..
55+
# Let's use specific commit for now
56+
# TODO: progress it once in a while
57+
commit=a7b291b882a8c6184219ccb84faae255ba96203a
58+
git clone --depth 1 https://github.com/bids-standard/bids-validator
59+
cd bids-validator
60+
git fetch --depth 1 origin $commit
61+
echo -e '#!/bin/sh\n'"$PWD/bids-validator/bids-validator-deno \"\$@\"" >| /tmp/bids-validator
62+
chmod a+x /tmp/bids-validator
63+
sudo mv /tmp/bids-validator /usr/local/bin/bids-validator
64+
which -a bids-validator
65+
bids-validator --help
66+
popd
67+
68+
- name: Display versions and environment information
69+
run: |
70+
echo $TZ
71+
date
72+
echo -n "npm: "; npm --version
73+
echo -n "node: "; node --version
74+
echo -n "bids-validator: "; bids-validator --version
75+
echo -n "python: "; python --version
76+
77+
# Checkout bids-examples
78+
- uses: actions/checkout@v4
79+
with:
80+
# For now use the forked repository with support for deno validator
81+
# from https://github.com/bids-standard/bids-examples/pull/435
82+
repository: yarikoptic/bids-examples
83+
ref: deno-validator
84+
path: bids-examples
85+
86+
- name: Mark known not yet to be deno-legit BIDS datasets
87+
run: touch {ds000117,ds000246,ds000247,ds000248,eeg_ds003645s_hed_demo,ieeg_motorMiller2007,ieeg_visual}/.SKIP_VALIDATION
88+
shell: bash
89+
working-directory: bids-examples
90+
91+
- name: Validate using bids-validator without migration
92+
run: ./run_tests.sh
93+
working-directory: bids-examples
94+
95+
- name: Migrate all BIDS datasets
96+
run: |
97+
for dataset in */dataset_description.json; do
98+
dataset_dir=$(dirname "$dataset")
99+
echo "Migrating $dataset_dir..."
100+
bst migrate all "$dataset_dir" --skip fix_inheritance_overloading --skip fix_tsv_entity_prefix || echo "Migration failed for $dataset_dir"
101+
done
102+
shell: bash
103+
working-directory: bids-examples
104+
105+
- name: Show migrated datasets diff
106+
run: git diff HEAD
107+
working-directory: bids-examples
108+
109+
- name: Validate all BIDS datasets using bids-validator after migration
110+
run: VALIDATOR_ARGS="--schema file://$PWD/../src/schema.json" bash ./run_tests.sh
111+
working-directory: bids-examples

0 commit comments

Comments
 (0)