Separate signals on neuralynx and improve streams names #1786
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This will fix #1778 and fix #1678.
NeuralynxRawIO on main is incorrectly grouping channels with fundamentally different acquisition configurations into the same stream. Eye-tracking channels and electrophysiology channels recorded at the same sampling rate would be merged together, despite having vastly different voltage ranges (100mV vs 1mV) and filter settings. This incorrect grouping has led to user errors and confusion. Additionally, stream names like "Stream (rate,#packet,t0): (32000.0, 300, 6382425039112)" are difficult to parse, type and use programmatically.
I've refactored stream identification to be based on acquisition hardware and signal processing parameters: sampling rate, input voltage range, gain, input inversion, and DSP filter configuration (low-cut/high-cut enabled status, frequencies, filter types, taps, delays). Examples:
Stream (rate,#packet,t0): (32000.0, 300, 6382425039112)
→stream0_32000Hz_100mVRange_DSPFilter0
Stream (rate,#packet,t0): (32000.0, 100, 6382425039112)
→stream1_32000Hz_1mVRange_DSPFilter1
Stream (rate,#packet,t0): (2000.0, 450, 6382425039112)
→stream2_2000Hz_100mVRange_DSPFilter2
These parameters define how signals are acquired and processed, sampling rate determines temporal resolution, voltage range and gain control amplitude scaling, and DSP filters alter frequency content. Channels with different values for any of these parameters represent fundamentally different signal types and cannot be meaningfully grouped together. Temporal properties (t_start, n_packets) have been removed from stream identification since they describe when/how much data was recorded, not the acquisition configuration itself. I display input voltage range in the stream name rather than gain because the range directly indicates the expected signal amplitude (e.g., 100mV for eye-tracking vs 1mV for ephys), which is more intuitive for users than a gain value that requires additional context to interpret. I am open to other suggestions in this front.
Since DSP filter configurations can be complex (combining multiple parameters like DSPLowCutFilterEnabled, DspLowCutFrequency, DSPHighCutFilterEnabled, DspHighCutFrequency, DspFilterType, DspDelayCompensation, etc.), I assign each unique filter configuration an enumerated ID and display it as DSPFilter{id} in the stream name. I choose to enumerate the filters because while they fully specify the acquisition stream and should be included in stream identification, displaying all filter parameters would make stream names too verbose (see example below).The detailed filter parameters are stored internally in
_dsp_filter_configurations
(kept private for now as an experimental feature, open to discussion if I want to keep it like that, expose via public API or through the annotations mechanism). Example structure:We also added type normalization in NlxHeader to ensure header values are proper Python types (booleans, integers, floats) rather than strings.
See the following table with our test suite examples to see how the stream naming will be affected: