MouseLand
Kilosort used to raise one of these errors every run: Err2: ValueError: Found array with 0 sample(s) (shape=(0, 61)) while a minimum of 1 is required by TruncatedSVD.
Describe the issue:
I used to get one of these errors when running Kilosort4 using the following code:
Err1- ValueError: n_samples=4 should be >= n_clusters=6.
Err2: ValueError: Found array with 0 sample(s) (shape=(0, 61)) while a minimum of 1 is required by TruncatedSVD.
#The attached image shows the channel map #The attached text file contains the data that is converted to binary to be used in the algorithm
The realDataMap4.bin file contains data of size (9X 6609993)
Reproduce the bug:
from kilosort import run_kilosort, DEFAULT_SETTINGS
settings = DEFAULT_SETTINGS
# ( path to drive if mounted: /content/drive/MyDrive/ )
settings['data_dir'] = 'C:/development/KioSort'
settings['n_chan_bin'] = 9
settings['fs']= 30000
settings['batch_size']= 60000
ops, st, clu, tF, Wall, similar_templates, is_ref, est_contam_rate, kept_spikes = \
run_kilosort(settings=settings,data_dir='C:/development/KioSort',
probe_name='C:/development/KioSort/ChannelMap4.mat',
filename='C:/development/OsortTextData/AlphaRS_RealData/Map4/realDataMap4.bin',
data_dtype ='int16')
Error message:
Err1:
File ~\anaconda3\Lib\site-packages\sklearn\base.py:633 in _validate_data
out = check_array(X, input_name="X", **check_params)
File ~\anaconda3\Lib\site-packages\sklearn\utils\validation.py:1072 in check_array
raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 61)) while a minimum of 1 is required by TruncatedSVD.
Err2: Err1- ValueError: n_samples=4 should be >= n_clusters=6.
Version information:
versions of: Kilosort 4 Python 3.12.4 Windows10 Cuda toolKit 12.6
Which version of Kilosort4 are you using? Can you please upload kilosort4.log from the results directory?
I don't know which one but I think I am using the last version v4.0.15 because I installed it before a week
@NadaRayann You're probably getting errors because your probe layout is not in the expected format. Values for 'xc' and 'yc' should be specified in microns. You probably need to swap 'xc' and 'yc' as well, currently it's a group of individual channels all at the same depth rather than multiple channels on a single shank. If that's how your probe is actually laid out, you should still swap so that 'xc' values are constant while 'yc' is changing for better processing. You can space 'yc' ~100 microns apart in that case so make sure the channels are still treated separately.
If you still run into problems after changing the probe layout, please upload a screenshot of what the KS4 GUI looks like after loading your data.
Actually I have a problem in specifying the xcoords and the ycoords since it is 3D (x,y,z). Also I can't specify the kcoords because I can't specify if the electrod that I use has single shank or multi shank. If you can help me in specifying the exact coordinates and the kcoords please do. The following description is our new design of the channels and you can check the image if something missed.
I'm using a directional electrode, called AlphaProbe (produced by Alpha Omega Company). My electrode contains 4 micro side contacts (with the same y axis coordinates), in addition to micro tip contact. the y axis(depth) distance between the micro tip contact and the 4 side contacts is 2.5 mm . The configuration of the side contacts is + , and the distance between the micro tip and each side contact is 275 microns. (Please see the attached image)
According to our electrode that we used in our experiment as I mentioned above we use a part of the data in Kilosort. The data is attached as a text file with its visualization.
lt1d2.042f0001.mat_44.0.txt
@jacobpennington Hi, we tried adjusting the ycoords and xcoords parameters, but we are still receiving the same error. "n_samples should be >= n_clusters" Could you please help us resolve this?
@Sana-Alshboul Kilosort does not support 3D coordinates. However, with contacts spaced that far apart the geometry won't matter anyway. You can just list the contacts as if they were on a single shank, like:
'xc': [0, 0, 0, 0],
'yc': [0, 200, 400, 600],
'kcoords': [0, 0, 0, 0]
@danafarah Please upload a screenshot of what the KS4 GUI looks like after loading the data, and the new kilosort4.log from running it with the new probe configuration.
@jacobpennington Thanks for answering. I use the following channel map setup: xcoord: [0, 0, 0, 0, 0] ycoord: [0, 100, 200, 300, 400] kcorrds: [1, 1, 1, 1, 1]
but still I got this error: File ~\AppData\Local\anaconda3\envs\kilosort\Lib\site-packages\sklearn\base.py:633 in _validate_data out = check_array(X, input_name="X", **check_params)
File ~\AppData\Local\anaconda3\envs\kilosort\Lib\site-packages\sklearn\utils\validation.py:1087 in check_array raise ValueError(
ValueError: Found array with 0 sample(s) (shape=(0, 61)) while a minimum of 1 is required by TruncatedSVD.
This is the log file: kilosort4.log
Can you please upload a screenshot of what the KS4 GUI looks like after loading your data?
The KS4 GUI is empty after loading the data because I got this message in the message log box: filter critical frequencies must be greater than 0
Traceback (most recent call last):
File "C:\Users\l.lahham\AppData\Local\anaconda3\envs\kilosort\Lib\site-packages\kilosort\gui\sorter.py", line 47, in run
if not results_dir.exists():
^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^ ^
AttributeError : 'NoneType' object has no attribute 'exists'
The first thing you should try is clicking "Reset GUI" to set all parameters to their defaults, then only change the things that you have to for your data (like number of channels and sampling frequency). For example, Th (universal) is set to 100, which will almost certainly result in no spikes ever being detected.
The following image shows the heatmap of the data. I know it is not correct because the power spectrum shows as diagonal line because number of channels doesn't match the data. But i am sure that the entered number of channels is correct because they are 5 and i visualized the binary file for 5 channels as shown in the previous comments.
I don't see anything indicating that the number of channels is incorrect. Clicking on "whitened" at the bottom-right of the heatmap will give you a better view of the data, though, since that will show the filtered and whitened version.
I would recommend trying the sorting again with nblocks = 0 to turn off drift correction (and setting sampling frequency appropriately as you had it before). I didn't catch that before, it's harder to see all the settings from scanning the log file. Drift correction will not work properly for that probe geometry, and it could result in some strange drift estimates that would cause the sorting to fail.
@jacobpennington I doute in generating the binary file, can you please check if the binary file that i use obeys the Kilosort rules. The file shall contains data of 5 channels. I generated the binary file by inserting the data of each channel in a separated row. So the file shall contains 5xn data shape, where n number of samples. Data.zip
After I setting the samplig rate to 44000 as the actual data sampling rate and nblocks =0, I got these figures.
@jacobpennington Please take a look on the previous comment as soon as possible
@Sana-Alshboul @NadaRayann @danafarah if you work for alpha-omega, I hope you are aware of Kilosort's licensing terms.
@marius10p Thank you in advance. could you please send me your email?
The Kilosort4 paper includes contact information. The github repository contains the license.