Skip to content

Add SortedSpikeGLXConverter and improve linking_sorted_data documentation #1449

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 2 commits into
base: main
Choose a base branch
from

Conversation

h-mayorquin
Copy link
Collaborator

@h-mayorquin h-mayorquin commented Jul 29, 2025

Adding data from SpikeGLX with multiple probes is a popular use case that currently lacks comprehensive documentation and tooling support. The existing documentation fo linking electrode metadata to sorted units has significant gaps (see
NeurodataWithoutBorders/helpdesk#97 (comment) plus personal discussions with @tabedzki), so I'm taking this opportunity to address both issues simultaneously by introducing the SortedSpikeGLXConverter and improving the overall documentation for this.

This PR introduces a new SortedSpikeGLXConverter that extends SpikeGLXConverterPipe to handle sorting data from multiple probes while maintaining proper unit-to-electrode linkage. Additionally, I've restructured and expanded the electrode linking documentation into a proper "how-to" guide which hopefully should be more clear.

This is a draft for discussion and I'm hoping to get feedback from the community.
@alejoe91, could you take a brief look at the kilosort section? I am not as familiar with the sorte as you are
@EthanKato, mentioned in the dandi-desk that he was converting data like this so your input would be valuable.
@simon-ball, this should address the use case you described in #1112.
I believe @yarikoptic and @manimoh may have encountered similar multi-probe scenarios as well and could benefit from this enhancement.

A link to the live documentation can be found here:
https://neuroconv--1449.org.readthedocs.build/en/1449/how_to/linking_sorted_data.html

@EthanKato
Copy link

My experience was just that using multiple sort streams in an NWBConverter and then running run_conversion() would result in nwb units overwriting itself. Instead, I am no longer using NWBConverter and call write_to_nwb on each data interface independently, writing to a processing module instead of to units.

Copy link

codecov bot commented Jul 29, 2025

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 87.27%. Comparing base (304691d) to head (f116eeb).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main    #1449      +/-   ##
==========================================
+ Coverage   87.04%   87.27%   +0.23%     
==========================================
  Files         145      146       +1     
  Lines        9681     9731      +50     
==========================================
+ Hits         8427     8493      +66     
+ Misses       1254     1238      -16     
Flag Coverage Δ
unittests 87.27% <100.00%> (+0.23%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/neuroconv/converters/__init__.py 100.00% <100.00%> (ø)
.../neuroconv/converters/sorted_spikeglx_converter.py 100.00% <100.00%> (ø)
...datainterfaces/ecephys/sortedrecordinginterface.py 100.00% <ø> (+37.50%) ⬆️
...tainterfaces/ecephys/spikeglx/spikeglxconverter.py 92.30% <100.00%> (+0.20%) ⬆️

... and 1 file with indirect coverage changes

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@yarikoptic
Copy link
Contributor

also attn @CodyCBakerPhD

@CodyCBakerPhD
Copy link
Contributor

and call write_to_nwb on each data interface independently, writing to a processing module instead of to units.

That can certainly work, but there are certain benefits to having a single combined table, such as easier nagivation to PSTH widgets such as Neurosift: https://neurosift.app/nwb?url=https://api.dandiarchive.org/api/assets/64c796e4-3935-4d43-b4ca-9d1df17a07c2/download/&dandisetId=000409&dandisetVersion=draft&tab=view:PSTH|/intervals/trials^/units

image

As such my usual method is to just directly adjust the unit IDs in the sorting extractor, such as in: https://github.com/catalystneuro/IBL-to-nwb/blob/main/src/ibl_to_nwb/datainterfaces/_ibl_sorting_extractor.py#L82

@h-mayorquin
Copy link
Collaborator Author

My experience was just that using multiple sort streams in an NWBConverter and then running run_conversion() would result in nwb units overwriting itself. Instead, I am no longer using NWBConverter and call write_to_nwb on each data interface independently, writing to a processing module instead of to units.

Yes, this is slightly different issue. This is how to link the sorting to the electrodes, I added a new how-to that explains how to handle more than one sorting interface here:

#1451

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants