ComputeIQMs crash on NaN, crashes without exporting existing outputs

[SLReeser]

What happened?

This is a continuation of #1392. I have three T1w runs for this participant I'm using to test with because it was one of the handfull of participants that failed for me. according to the log file, procesing is occuring sequentially in order by run, and it is failing on the third run. No data (figures, json or html files are generated) for runs already processed internally at the time. With this version (as opposed to 24 or 25) I do now get an error that might be helpful. ComputeQI2 is outputing a NaN and erroring on output.

What command did you use?

#!/bin/bash

if [ $# -lt 1 ]; then
   # Prompt proper calling format
   echo " bash Q1_mriqc.sh subjects.txt"
   exit
fi

# Save input SubID
SUBJECTS=`cat ${1}`

TRUE_PATH=$(pwd -P) # Use this so that shortcuts from your home directory aren't an issue
CODE_DIR=$(dirname $TRUE_PATH)
PROJECT_HOME=$(dirname $CODE_DIR)

input_dir=${PROJECT_HOME}/nifti
output_dir=${PROJECT_HOME}/derivatives/MRIqc_anal
container_loc=/projects3/singularity_images/MRIqc/mriqc_experimental.sif
#container_loc=/projects3/singularity_images/MRIqc/mriqc-24.0.2.sif # Doesn't complete for some participants errors out with no output. 
#container_loc=/projects3/singularity_images/MRIqc/mriqc_25.0.0rc0.sif # same.

for ID in ${SUBJECTS}; do

	echo "------------"
	echo ${ID}
	echo "------------"
	date
	echo "input data:           ${input_dir}"
	echo "output data:          ${output_dir}"
	echo "Container location:   ${container_loc}"
	echo "participant:          ${ID}"
	echo "data submission:      no submission" 
	echo "PROJECT_HOME          ${PROJECT_HOME}"
	date
	
	apptainer run --cleanenv --bind ${input_dir}:/data --bind ${output_dir}:/out ${container_loc} /data /out participant --participant-label ${ID} --no-sub --species human --participant-label ${ID} --modalities T1w --float32 --bids-database-wipe --no-datalad-get --crashfile-format txt --notrack --verbose-reports -vvv; \
	

	echo "end ${ID}"
	date
done

What version of the software are you running?

mriqc_experimental.sif

How are you running this software?

Singularity

Is your data BIDS valid?

Yes

Are you reusing any previously computed results?

No

Please copy and paste any relevant log output.

# From log file
# mriqc-20250512-140615_eb2cf8d4-995c-4217-8b1b-33938058a273.log
# ----------------------------------------------------------------
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.__init__(None, /gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/AirMaskWorkflow/2dff733bcff2fd9da46f1425e34fdd683c4a704d/ArtifactMask/sub-CUBS702421_ses-01_acq-QALAS_run-03_T1w_conformed.nii_hat.nii.gz, True, 4194304, 32768, 1048576, True)
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.close()
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.__init__(None, /gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/2dff733bcff2fd9da46f1425e34fdd683c4a704d/conform/sub-CUBS702421_ses-01_acq-QALAS_run-03_T1w_conformed.nii.gz, True, 4194304, 32768, 1048576, True)
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.seek(352)
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.seek(352)
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.close()
2025-05-12 14:11:31 | INFO     | nipype.workflow  | [Node] Finished "ComputeQI2", elapsed time 0.286074s.
2025-05-12 14:11:31 | DEBUG    | nipype.workflow  | Saving results file: '/gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/ComputeIQMs/2dff733bcff2fd9da46f1425e34fdd683c4a704d/ComputeQI2/result_ComputeQI2.pklz'
2025-05-12 14:11:31 | WARNING  | nipype.workflow  | Storing result file without outputs
2025-05-12 14:11:31 | WARNING  | nipype.workflow  | [Node] Error on "mriqc_wf.anatMRIQC.ComputeIQMs.ComputeQI2" (/gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/ComputeIQMs/2dff733bcff2fd9da46f1425e34fdd683c4a704d/ComputeQI2)
2025-05-12 14:11:31 | DEBUG    | nipype.utils     | Loading pkl: /gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/ComputeIQMs/2dff733bcff2fd9da46f1425e34fdd683c4a704d/ComputeQI2/result_ComputeQI2.pklz
2025-05-12 14:11:31 | ERROR    | nipype.workflow  | Node ComputeQI2.a1 (taskid=49) crashed: /out/logs/crash-20250512-141131-salarsen-ComputeQI2.a1-c366869e-087d-499a-afd7-e3bed4b0e4ab.txt
2025-05-12 14:11:31 | DEBUG    | indexed_gzip.indexed_gzip | _IndexedGzipFile.seek(352)
2025-05-12 14:11:31 | DEBUG    | matplotlib.font_manager | findfont: Matching sans\-serif:style=normal:variant=normal:weight=normal:stretch=normal:size=12.0.
2025-05-12 14:11:31 | DEBUG    | matplotlib.font_manager | findfont: score(FontEntry(fname='/opt/conda/lib/python3.11/site-packages/matplotlib/mpl-data/fonts/ttf/DejaVuSansDisplay.ttf', name='DejaVu Sans Display', style='normal', variant='normal', weight=400, stretch='normal', size='scalable')) = 10.05
2025-05-12 14:11:31 | DEBUG    | matplotlib.font_manager | findfont: score(FontEntry(fname='/opt/conda/lib/python3.11/site-packages/matplotlib/mpl-data/fonts/ttf/cmsy10.ttf', name='cmsy10', style='normal', variant='normal', weight=400, stretch='normal', size='scalable')) = 10.05
2025-05-12 14:11:31 | DEBUG    | matplotlib.font_manager | findfont: score(FontEntry(fname='/opt/conda/lib/python3.11/site-packages/matplotlib/mpl-data/fonts/ttf/STIXGeneral.ttf', name='STIXGeneral', style='normal', variant='normal', weight=400, stretch='normal', size='scalable')) = 10.05

# From Crash file
# crash-20250512-141131-salarsen-ComputeQI2.a1-c366869e-087d-499a-afd7-e3bed4b0e4ab.txt
# -------------------------------------------
Node: mriqc_wf.anatMRIQC.ComputeIQMs.ComputeQI2
Working directory: /gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/ComputeIQMs/2dff733bcff2fd9da46f1425e34fdd683c4a704d/ComputeQI2

Node inputs:

air_msk = /gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/AirMaskWorkflow/2dff733bcff2fd9da46f1425e34fdd683c4a704d/ArtifactMask/sub-CUBS702421_ses-01_acq-QALAS_run-03_T1w_conformed.nii_hat.nii.gz
in_file = /gnostic/CUBS/BIDS/code/bin/work/mriqc_wf/anatMRIQC/2dff733bcff2fd9da46f1425e34fdd683c4a704d/conform/sub-CUBS702421_ses-01_acq-QALAS_run-03_T1w_conformed.nii.gz

Traceback (most recent call last):
  File "/opt/conda/lib/python3.11/site-packages/mriqc/engine/plugin.py", line 64, in run_node
    result['result'] = node.run(updatehash=updatehash)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 527, in run
    result = self._run_interface(execute=True)
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 645, in _run_interface
    return self._run_command(execute)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/conda/lib/python3.11/site-packages/nipype/pipeline/engine/nodes.py", line 771, in _run_command
    raise NodeExecutionError(msg)
nipype.pipeline.engine.nodes.NodeExecutionError: Exception raised while executing Node ComputeQI2.

Traceback:
	Traceback (most recent call last):
	  File "/opt/conda/lib/python3.11/site-packages/nipype/interfaces/base/core.py", line 397, in run
	    runtime = self._run_interface(runtime)
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/mriqc/interfaces/anatomical.py", line 377, in _run_interface
	    qi2, out_file = art_qi2(imdata, airdata)
	                    ^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/mriqc/qc/anatomical.py", line 497, in art_qi2
	    kde_skl = KernelDensity(kernel='gaussian', bandwidth=4.0).fit(modelx[:, np.newaxis])
	              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/sklearn/base.py", line 1351, in wrapper
	    return fit_method(estimator, *args, **kwargs)
	           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/sklearn/neighbors/_kde.py", line 226, in fit
	    X = self._validate_data(X, order="C", dtype=np.float64)
	        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/sklearn/base.py", line 633, in _validate_data
	    out = check_array(X, input_name="X", **check_params)
	          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
	  File "/opt/conda/lib/python3.11/site-packages/sklearn/utils/validation.py", line 1003, in check_array
	    _assert_all_finite(
	  File "/opt/conda/lib/python3.11/site-packages/sklearn/utils/validation.py", line 126, in _assert_all_finite
	    _assert_all_finite_element_wise(
	  File "/opt/conda/lib/python3.11/site-packages/sklearn/utils/validation.py", line 175, in _assert_all_finite_element_wise
	    raise ValueError(msg_err)
	ValueError: Input X contains NaN.
	KernelDensity does not accept missing values encoded as NaN natively. For supervised learning, you might want to consider sklearn.ensemble.HistGradientBoostingClassifier and Regressor which accept missing values encoded as NaNs natively. Alternatively, it is possible to preprocess the data, for instance by using an imputer transformer in a pipeline or drop samples with missing values. See https://scikit-learn.org/stable/modules/impute.html You can find a list of all estimators that handle NaN values at the following page: https://scikit-learn.org/stable/modules/impute.html#estimators-that-handle-nan-values

Additional information / screenshots

There are some suggestions are at the bottom of the crash report for how NaNs might be handeled included in the log window above that may be helpful.