Binary file format and pre-processing issue

[manfredjoswald]

Hello, I have problems with converting 64-channel data recorded with Intan from an unsigned uint16 binary file to int16 format. The uint16 file is entered around 32768 with limits of 0 and 65535, so when I convert the data in Matlab to int16 with 'int16(a_data)' I just saturate all the data above 32768. However, the KiloSort output still provides me with spikes, a lot of them appear to be noise though. The traceView then displays alternate waveform and saturated straight line sections (see the recent Issue I posted to Phy). When I subtract 32768 from my uint16 data I can generate a binary int16 file with a mean around zero that is not saturated. However, now the same KiloSort settings do not detect any spikes at all, and also not if I reduce the spike threshold detection parameter setting. So my first question is if the int16 binary file processed by KiloSort can contain negative values or just 0 to 32768? And if the values can range from -32768 to +32768, what parameters in the Configuration file should I change to detect spikes. Another approach to zero data is to filter it first with the Matlab filtfilt command. For this I have to convert first to double precision format and then back to int16, but this step is very slow. I assume that is why the integer data is filtered during pre-processing before Spike detection in KiloSort. I am not familiar with the data format used in the pre-processing steps (DATA now has a 3rd dimension). If I use the same filter parameters should the output then be identical to what I obtain with the filtfilt function? Can somebody tell me what file Phy uses for plotting in the TraceView window? Is that the unfiltered data? It does very different to the filtfilt output in double precision format. And finally, does it make sense to use the int16 binary file of the data that is already filtered (with filtfilt) as input for KiloSort, or do I have to remove the filter sections then from the pre-processing m-file of the KiloSort package? I am grateful for any comments towards these questions.

[manfredjoswald]

All good, the problem with no spikes being detected from mean-zero int16 binary file was fixed by setting ops.initialize to 'no' instead of 'fromData'. I now get the same result whether I filter the data as specified in the preprocessData.m file or instead use the slower filtfilt function with the same parameters on the double precision data. ~Manfred