Tdt format compatibility

[JulesLebert]

Hi, I am not sure if that should be a pull request or just a question on my end for the compatibility spikeinterface/neo. If these changes are too specific to my data format, I will just keep using my forked repository of Neo. I am just following @samuelgarcia suggestion to continue this discussion on the neo side (SpikeInterface/spikeinterface#1138)

There are two changes in this pull request:

The first one is that the regular expression to find the .sev files was not working for me, even for the test data (https://gin.g-node.org/NeuralEnsemble/ephy_testing_data)

With changes in the regular expression, I was able to load the test data:

The other changes are specific to my data (but are also compatible with the test data).
My tdt data are in the following format:

This is one recording session (or one neo segment), where BB_2 and BB_3 are one 32 channel array and BB_4 and BB_5 another 32 channel array. Both 32 channel arrays are split into two streams (or TDT ‘stores’) of 16 channels.

Some changes I had to do to load the data in neo are a few changes in the lines following the regular expression. Because the regular expression was returning the .sev files from all stores.

The other change is an addition in the conditions of the mask for the tsq data. I'm not sure why, but even though the TankEvType of the different stream in info_channel_group corresponds to the default value of EVTYPE_STREAM (33025), the ‘evtype’ in the tsq files for my data containing the analog signals are 33041 instead of 33025. Therefore, adding another condition in the mask allows me to load the data (otherwise data_index returns an empty array)

After all these changes, test data can be loaded as usual:

As well as my own tdt data:

And when I compare the Neo data to the data loaded by the tdt python package (https://www.tdt.com/docs/sdk/offline-data-analysis/offline-data-python/00_Intro/) they are the same:

With all this in mind, and following my question on the spikeinterface repository, is there a way to load the different streams into one, which would allow me to spike sort everything in one go through spikeinterface instead of having to run 4 different spike sorter with one for each stream?

[JuliaSprenger]

Hi @JulesLebert Thanks for your detailed description of the issue and your fix for it! I added a few suggestions. Can you have a look and see if you can still load your files including my suggestions?

[JulesLebert]

Hi, sorry for the late response. Here are some answers to the suggestions.

[JuliaSprenger]

Hi @JulesLebert, any news on this?

[JulesLebert]

Hi @JuliaSprenger I have just made the suggested changes and it seems to be compatible with both the test data and my data

[JuliaSprenger]

Hi @JulesLebert. Thanks for including the comments. I made some minor reformatting. Looks ready to merge for me now.