[ENH] Formalize presence of optional logs/ folder

[yarikoptic]

It is quite often desired to store logs, e.g. simply stdout/stderr from invocation of BIDS converters or other tools which were used to produce/change content in current BIDS datasset. They quite often provide ultimate provenance information to troubleshoot odd or incorrect results. But researchers, in my personal opinion, undervalue logs! But hinting them on "standard" location for them, I think we could inspire more of relevant to dataset provenance metadata being collected and shared.

TODOs:

• ?Danger: might leak sensitive metadata, so may be wording should be adjusted to mention that they should be inspected/sensored? WDYT? I think it is fine/assumed

Shameless plug: consider using con-duct (https://github.com/con/duct) for your "logging needs".

Note that I think that ideally the logs/ should not include "derivative" data, such as e.g. output of bids-validator. Those outputs should go under derivatives/bids-validator-{version}/output.json or alike. (It might be the only reasonable "derivative" to recommend bundling with any "raw" or derived BIDS dataset).

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 86.87%. Comparing base (7ddb7be) to head (1a50063).
Report is 5 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #1962   +/-   ##
=======================================
  Coverage   86.87%   86.87%           
=======================================
  Files          16       16           
  Lines        1410     1410           
=======================================
  Hits         1225     1225           
  Misses        185      185           

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[effigies]

I'm fine with this. Up to now I probably would put them under code/ but that's not really the right place.

This is, IMO low-priority to standardize, since the logs themselves are not standardized, so this doesn't make them substantially more legible to tooling. But it doesn't harm anything that I can see and it's unlikely to have another use.

[yarikoptic]

ping @Remi-Gau @ericearl , WDYT? I am seeking for more approvals ;)

[ericearl]

I read and understood the src/common-principles.md changes and they lgtm. I assume the rest of the changes to the schema which look probably right are fine.

One non-blocking question though: If they are logs from running code, why wouldn't you want the logs/ subdirectory under the code/ subdirectory? Imagine a dataset with no code/ subdirectory, but a logs/ subdirectory is present. It leaves the end-user wondering a bit what's going on. I personally usually put my logs from code into the code/subdirectory as-is.

[yarikoptic]

I think it is valuable to logs/ from code/ for a number of reasons:

• logs/ would primarily be machine generated (although could be some logs from "operator"), whenever code/ is human (well, might be AI ;)) produced
• logs/ might be produced by commands which are not under code/ , e.g. straight invocations of bet, or bids-validator or anything else.
• could be .gitignored
  • I would cherish code/ content and likely ignore logs/ until I need to investigate.
• could be made into git submodule and thus not part of the dataset for some reasons (size, leak of sensitive information, etc)
• I, as a linux user with logs going under /var/log and some ~/.cache while code under /usr/bin etc, personally would not expect logs be mixed into folder with tools