CaImAn summary images bug

[h-mayorquin]

Currently, CaImAn stores the image mean as the array b:

Extracted here:

roiextractors/src/roiextractors/extractors/caiman/caimansegmentationextractor.py

Lines 135 to 140 in 26fafef

	def _summary_image_read(self):
	"""Read summary image mean."""
	if self._dataset_file["estimates"].get("b"):
	FOV_shape = self._dataset_file["params"]["data"]["dims"][()]
	b_sum = self._dataset_file["estimates"]["b"][:].sum(axis=1)
	return np.array(b_sum).reshape(FOV_shape, order="F")

And assigned here:

roiextractors/src/roiextractors/extractors/caiman/caimansegmentationextractor.py

Lines 51 to 52 in 26fafef

	self._image_mean = self._summary_image_read()
	self._sampling_frequency = self._dataset_file["params"]["data"]["fr"][()]

However, the documentation states that b is the spatial factor of the background component in the two-photon CNMF decomposition (the corresponding temporal factor is f):

https://caiman.readthedocs.io/en/latest/Getting_Started.html#result-interpretation

Averaging b alone does not produce a meaningful summary image; even the mean background would require multiplying b by f and that is still missing the ROIs contribution.

In Suite2P, by contrast, the mean image is defined as the pixel-wise mean across all registered frames (see meanImg):

https://suite2p.readthedocs.io/en/latest/outputs.html

And is calculated in exactly that way:
https://github.com/MouseLand/suite2p/blob/ddb8efd2ea3bae6f9a3a9253dc0ac336cf8721b2/suite2p/detection/detect.py#L143-L146

So, the what we store in CaImAn as a summary image is at odds with what we store in Suit2p. They are not the same thing. Moreover, there is an equivalent quantity in CaImAn (that I am not sure how often is available), mn:

https://github.com/flatironinstitute/CaImAn/blob/881e627adf951dde25d3839953c98acf6b4adab0/caiman/summary_images.py#L756-L809

Because the main goal of this library is to write results to NWB, I would prefer to store mn whenever it is available and omit the mean image otherwise. I think this is better than saving a potentially misleading average of b.

Thoughts? Am I wrong?

[h-mayorquin]

I guess the critical think is, when we are storing summary images, specifically the mean. What information do we want to store?

I am looking at the future ndx-microscopy:
https://github.com/catalystneuro/ndx-microscopy/blob/1f15204f66fc0180c2919af88ad01168a698e38f/spec/ndx-microscopy.extensions.yaml#L328-L331

And it is not very clear. I think that the central case is the mean over the samples as Suit2p does but maybe this is a more general concept. It would be good to get your input, @alessandratrapani.

[alessandratrapani]

Because the main goal of this library is to write results to NWB, I would prefer to store mn whenever it is available and omit the mean image otherwise. I think this is better than saving a potentially misleading average of b.

Thoughts? Am I wrong?

You are totally right, it should be the mean image (across frames), or maximum projection or correlation image or pnr image

[h-mayorquin]

I will add a keyword that enables this behavior (adding the mean of the background ROIs as a summary image) and throw a warning that it will be set to False in a while. That way we won't break backwards compatibility.