in CLI mode and online provide alternative formattings and grouppins

[yarikoptic]

Might want to become separate issues. Inspired by

• In --json output, include "hed string" and point to multiple rows when applicable to HED error bids-standard/bids-validator#175 (comment)

Since Python one is used in online invocation, decided to file here

• For the .txt I think it should just return http response with correct content-type so browser could show / render it without demanding to be saved
• Would be nice to get some drop down to select an alternative format, e.g. json or yaml where such records would be provided in a machine readable format...

In web and CLI:

• ideally there should be some web UI to visualize them neatly, e.g. to group by type of error . E.g in our case we had 200 errors of 20 "types" from two .tsv file . So looking at 200 of ungroupped errors is "hard". Having json records from bids-validator, and stripping embedded location (should be a separate field):

$ jq '.issues.issues[] | select(.code == "HED_ERROR")' < derivatives/bids-validator/deno-bids-validator.json | grep issueMessage | sort | sed -e 's,TSV line.*,,g' | uniq -c | sort -n | nl
     1        2   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Image, Task-event-role/Experimental-stimulus, Pathname/task-social/cue/runtype-cognitive/scl/l008.png)\".
     2        2   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Image, Task-event-role/Experimental-stimulus, Pathname/task-social/cue/runtype-cognitive/scl/l022.png)\".
     3        2   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Image, Task-event-role/Experimental-stimulus, Pathname/task-social/cue/runtype-cognitive/scl/l047.png)\".
     4        2   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"((Task-event-role/Experimental-stimulus), (Categorical-level-value/High))\".
     5        2   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"((Task-event-role/Experimental-stimulus), (Categorical-level-value/Low))\".
     6        2   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"((Task-event-role/Experimental-stimulus), (Categorical-level-value/Medium))\".
     7        3   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Action/Perceive\".
     8        3   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Image, Task-event-role/Experimental-stimulus, Pathname/task-social/outcomerating/task-cognitive_scale.png)\".
     9        3   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Intended-effect, Action/Think/Discriminate, Categorical-judgment-value/Different)\".
    10        3   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Task-action-type/Incorrect-action\".
    11        3   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Visual-presentation, Task-stimulus-role/Cue)\".
    12        4   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Data-value/Quantitative-value/Item-index/10\".
    13        4   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Data-value/Quantitative-value/Item-index/11\".
    14        4   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Data-value/Quantitative-value/Item-index/12\".
    15        6   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Action/Think/Encode\".
    16        6   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Data-source-type/Participant-generated, Data-value/Quantitative-value/Item-interval/0.0)\".
    17       12   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Condition-variable/cognitive, Image, Action/Think/Discriminate)\".
    18       12   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Task-stimulus-role/Cue, Categorical-level-value/Low)\".
    19       64   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"Organizational-property/Task/tomspunt\".
    20       64   "issueMessage": "ERROR: [TAG_EXPRESSION_REPEATED] Duplicate tag - \"(Task-stimulus-role/Cue, Action/Perform/Read)\".

Even those grouppings could be done in JSON/YAML forms, e.g. could be a dict of

{
   "issueMessage":
      "Duplicate tag - \"(Condition-variable/cognitive, Image, Action/Think/Discriminate)\"": [
          # Full list of records associated with that `issueMessage`
      ]
}

[VisLab]

I agree this should be a long-term improvement. However, we strongly recommend that users validate one events file and one JSON sidecar using the online validator (or --- yes it will be coming soon --- the CLI). Most of the time, resolving the errors in one typical events file will resolve them in all and this won't come up.

[yarikoptic]

However, we strongly recommend that users validate one events file and one JSON sidecar using the online validator

note:

• life is too short - we all should enjoy it
• we have

❯ ls sub-0*/**/*events.tsv | wc -l
4221

files ;)