Might be a corrupt spike_clusters.npy file? Double check that you have spike_templates.npy and, if so, rename spike_clusters.npy to spike_clusters.npy~ and try again?
I have the same problem with one of my data sets, I did used the zero_padding version (sometimes it happens and that so far always was a solution) but this time I can't open the dataset. I changed the spike_clusters.npy to spike_clusters.npy~ but still get the same error message.
I did rerun the same file with kilosort2.5 (without the zero_padding) but it doesn't let me open it with phy.
Is there anything I can do to be able to curate this dataset ?
(phy2) X:\Data\Mice\VR\Janna\DATA_BRAINSTEM\PR\JA0110\20210408\001\NP>phy template-gui params.py
←[33m12:01:54.909 [W] model:1153 File D:\TEMP_KILOSORT\temp_wh.dat does not exist.←[0m
←[33m12:01:55.362 [W] model:544 Unreferenced clusters found in templates (generally not a problem)←[0m
←[33m12:01:55.581 [W] model:566 Unreferenced clusters found in spike_clusters (generally not a problem)←[0m
←[33m12:01:55.628 [W] model:54 1851/96100 values are nan in \angelakilab.cns.nyu.edu\server3\Data\Mice\VR\Janna\DATA_BRAINSTEM\PR\JA0110\20210408\001\NP\similar_templates.npy, replacing by zero.←[0m
←[33m12:01:55.643 [W] model:158 Path D:\TEMP_KILOSORT\temp_wh.dat does not exist, trying ephys.raw filename.←[0m
←[33m12:01:55.643 [W] model:161 Error while loading data: File D:\TEMP_KILOSORT\ephys.raw.dat not found.←[0m
c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phylib\io\model.py:771: RuntimeWarning: invalid value encountered in greater_equal
peak_channels = np.nonzero(amplitude >= self.amplitude_threshold * max_amp)[0]
←[31m12:01:57.253 [E] init:62 An error has occurred (AssertionError):
Traceback (most recent call last):
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main
return _run_code(code, main_globals, None,
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\jma18\AppData\Local\Continuum\miniconda3\envs\phy2\Scripts\phy.exe_main.py", line 7, in
sys.exit(phycli())
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in call
return self.main(*args, **kwargs)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main
rv = self.invoke(ctx)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke
return callback(*args, **kwargs)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\apps_init.py", line 159, in cli_template_gui
template_gui(params_path, **kwargs)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui
gui = controller.create_gui()
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui
self.supervisor.attach(gui)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 941, in attach
self._create_views(
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views
gui, data=self.cluster_info, columns=self.columns, sort=sort)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info
return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in
return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info
out[key] = func(cluster_id)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label
return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
out = f(*args, **kwargs)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel
channel_ids = self.get_best_channels(cluster_id)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
out = f(*args, **kwargs)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels
template = self.model.get_template(template_id)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template
return self._get_template_dense(template_id, channel_ids=channel_ids)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in get_template_dense
channel_ids, amplitude, best_channel = self._find_best_channels(template)
File "c:\users\jma18\appdata\local\continuum\miniconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
assert best_channel in channel_ids
AssertionError
←[0m
I have another really important recording where I get the same error message. I wasn't able to solve the problem a few month ago, at some point I gave up and accepted that I lost the recording. The new one is really important and I can't give up on it. I tried the zero_padding kilosort version, I rerun kilosort with it multiple times. I deleted the .phy folder nothing worked. Do you have any idea where this assertion Error comes from and what I can do to get around it ?
(phy2) W:\20211207_JA0144_ADN_red_hstripe1_recording_g0\20211207_JA0144_ADN_red_hstripe1_recording_g0_imec0>phy template-gui params.py
←[33m23:15:33.704 [W] model:1153 File D:\TEMP_KILOSORT\temp_wh.dat does not exist.←[0m
←[33m23:15:35.806 [W] model:545 Unreferenced clusters found in templates (generally not a problem)←[0m
←[33m23:15:37.407 [W] model:567 Unreferenced clusters found in spike_clusters (generally not a problem)←[0m
←[33m23:15:37.507 [W] model:54 4308/131044 values are nan in \dalab43w.cns.nyu.edu\E\20211207_JA0144_ADN_red_hstripe1_recording_g0\20211207_JA0144_ADN_red_hstripe1_recording_g0_imec0\similar_templates.npy, replacing by zero.←[0m
←[33m23:15:37.507 [W] model:158 Path D:\TEMP_KILOSORT\temp_wh.dat does not exist, trying ephys.raw filename.←[0m
←[33m23:15:37.507 [W] model:161 Error while loading data: File D:\TEMP_KILOSORT\ephys.raw.dat not found.←[0m
←[31m23:15:40.303 [E] init:62 An error has occurred (AssertionError):
Traceback (most recent call last):
File "c:\users\lab.conda\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main
"main", mod_spec)
File "c:\users\lab.conda\envs\phy2\lib\runpy.py", line 85, in run_code
exec(code, run_globals)
File "C:\Users\lab.conda\envs\phy2\Scripts\phy.exe_main.py", line 7, in
sys.exit(phycli())
File "c:\users\lab.conda\envs\phy2\lib\site-packages\click\core.py", line 1025, in call
return self.main(*args, **kwargs)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\click\core.py", line 955, in main
rv = self.invoke(ctx)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\click\core.py", line 1517, in invoke
return process_result(sub_ctx.command.invoke(sub_ctx))
File "c:\users\lab.conda\envs\phy2\lib\site-packages\click\core.py", line 1279, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\click\core.py", line 710, in invoke
return callback(*args, **kwargs)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\click\decorators.py", line 18, in new_func
return f(get_current_context(), *args, **kwargs)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\apps_init.py", line 159, in cli_template_gui
template_gui(params_path, **kwargs)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui
gui = controller.create_gui()
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui
self.supervisor.attach(gui)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach
gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views
gui, data=self.cluster_info, columns=self.columns, sort=sort)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info
return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in
return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info
out[key] = func(cluster_id)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label
return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
out = f(*args, **kwargs)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel
channel_ids = self.get_best_channels(cluster_id)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
out = f(*args, **kwargs)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels
template = self.model.get_template(template_id)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template
return self._get_template_dense(template_id, channel_ids=channel_ids)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in get_template_dense
channel_ids, amplitude, best_channel = self._find_best_channels(template)
File "c:\users\lab.conda\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
assert best_channel in channel_ids
AssertionError
←[0m
I went through a bunch of other threads and was able to solve some of the assertion errors but I start running in really persistant recordings/errormessages. Most of the time I can prevent the assertion error when I turn of the registration (changing ops.nblocks to 0) but I am a bit concerned about that hack and unfortunately I am having assertion errors more than once in a while. I am using NP2.0 and have the problem independently of doing either hstripe or single shank recordings. I tried 3 different NP2.0 probes and I have this with some recordings with all of the probes at some point. I am not sure what exactly might cause the insertion error, does anyone know what this could be about ? I am happy to try out different things or to provide my recording (it is a short one , 16 min). I also went back and checked the recording to make sure there is no saturating noise in the recording but it doesn't seem to make a difference.
KiloSort copied to clipboard
cortex-lab
