phy icon indicating copy to clipboard operation
phy copied to clipboard

Assertion Error when opening spike sorted file

Open Idavr opened this issue 3 years ago • 43 comments

Hi,

I recently ran a recording through Kilosort2.5, where it reported to find spikes and good units (if few), but when I try to open it up in phy I get this error:

[33m16:07:42.861 [W] model:545            Unreferenced clusters found in templates (generally not a problem)[0m
[33m16:07:42.938 [W] model:567            Unreferenced clusters found in spike_clusters (generally not a problem)[0m
[33m16:07:42.965 [W] model:54             4279/40000 values are nan in C:\Users\idavalik\Data\Neuropixels\Socal_track_day5_g0\Socal_track_day5_g0_imec0\similar_templates.npy, replacing by zero.[0m
[31m16:07:43.826 [E] __init__:62          An error has occurred (AssertionError):
Traceback (most recent call last):
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\idavalik\AppData\Local\Continuum\anaconda3\envs\phy2\Scripts\phy.exe\__main__.py", line 7, in <module>
    sys.exit(phycli())
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in __call__
    return self.main(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main
    rv = self.invoke(ctx)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke
    return callback(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\__init__.py", line 159, in cli_template_gui
    template_gui(params_path, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui
    gui = controller.create_gui()
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui
    self.supervisor.attach(gui)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach
    gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views
    gui, data=self.cluster_info, columns=self.columns, sort=sort)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info
    return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in <listcomp>
    return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info
    out[key] = func(cluster_id)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label
    return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
    out = f(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel
    channel_ids = self.get_best_channels(cluster_id)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached
    out = f(*args, **kwargs)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels
    template = self.model.get_template(template_id)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template
    return self._get_template_dense(template_id, channel_ids=channel_ids)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in _get_template_dense
    channel_ids_, amplitude, best_channel = self._find_best_channels(template)
  File "c:\users\idavalik\appdata\local\continuum\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
    assert best_channel in channel_ids
AssertionError
[0m

The recording was very noisy and I fear/suspect some probe damage, but I have had no problems opening noisy files before.

Idavr avatar Nov 10 '20 15:11 Idavr

I also get this error recently after running kilosort on a full session with a few noisy channels disabled. If I run it on a 10-second segment of the data I don't get this error in phy.

peterzh avatar Nov 12 '20 11:11 peterzh

So no workaround to look at the entire session? Anyone knows the reason for this?

Idavr avatar Nov 13 '20 09:11 Idavr

I'm not sure if Kilosort2 rewrites params.py after sorting for a second or third time. Did you try to delete all files generated by Kilosort2.5 and start over again? It's working in my case.

JoseGuzman avatar Nov 13 '20 12:11 JoseGuzman

I tried re-running after deleting any of the files generated by KS2.5 but I still get this error. I don't know the exact cause of this error but I suspect it has something to do with the fact that in my dataset (and maybe @Idavr as well?), a few channels are disabled. Then assert best_channel in channel_ids fails because the best channel for one cluster is mis-assigned to one of the disabled channels. @marius10p do you think this is possible?

peterzh avatar Nov 13 '20 13:11 peterzh

I've also tried re-running it multiple times, getting the same error. Since this happened after something funky happened with the channels during the actual recording (the above mentioned noise) I am unable to open any sessions done after this weirdness, even if the raw stream of the recording does not have the same kind of noise. There are still very - very - few good units being detected by KS2.5, so I can not say whether that is a confounding factor but when there has been no spikes or units previously I haven't been able to run KS at all. A sudden channel mislabeling (for whatever reason) would make sense, but I am very interested to hear what Marius has to say on the subject.

Idavr avatar Nov 13 '20 14:11 Idavr

I think the key point is when using probes with blanked channels. Now when I use old probes with blanked channels, Phy show the channels numbered from zero to (n - number of subtracted channels). For example, if I subtract 2 and 3 from a 10-line probe (form 0-9), it will show probes between 0 and 7 (previously 2 and 3 were simply not shown). I'm not sure if this is related to this issue in Kilosort2.5

JoseGuzman avatar Nov 13 '20 18:11 JoseGuzman

If someone was able to send me a full dataset (even tiny) that either does not open, or that has mislabeled channels (with indications of where the actual labels should be), I could have a look

rossant avatar Nov 13 '20 20:11 rossant

@rossant Appreciate it very much! I could send you one of mine to take a look at? How would you want it sent?

Idavr avatar Nov 14 '20 15:11 Idavr

Any file-sharing like Dropbox, at cyrille.rossant at gmail.com

Le sam. 14 nov. 2020 à 16:28, Idavr [email protected] a écrit :

@rossant https://github.com/rossant Appreciate it very much! I could send you one of mine to take a look at? How would you want it sent?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/cortex-lab/phy/issues/1051#issuecomment-727223292, or unsubscribe https://github.com/notifications/unsubscribe-auth/AAO2GVZAV2SB3SGSA7YSR7LSP2O3FANCNFSM4TQZDDGA .

rossant avatar Nov 14 '20 16:11 rossant

Sent.

Idavr avatar Nov 14 '20 20:11 Idavr

I'm now also getting a similar error trying to run phy on a KS 2.5 sorted session where a reference channel on an NPx probe was disabled. Any insights since the last posting?

Hankslab-ABG avatar Nov 19 '20 23:11 Hankslab-ABG

Exactly, it happens when some channels are disconnected.

JoseGuzman avatar Nov 20 '20 13:11 JoseGuzman

Update: It does seem like there is some kind of issue with the KS2.5 output, as even if I specify the correct channels the params.py file gets saved with 383 channels (instead of 385) and the dat-path is set to be the temp_wh.dat file instead of the raw .bin file. Running the same files through KS2 I had no trouble accessing them with phy and params.py was saved correctly. @Hankslab-ABG have you checked the params.py file for inconsistencies compared to when you could run phy on your data?

Idavr avatar Nov 21 '20 09:11 Idavr

@ldavr I think that works as intended. The dat path should be set to the drift corrected file, otherwise you'll see the raw spikes moving with drift. The 383 channels are probably the connected ones on your probe, although it should normally be 383.

marius10p avatar Nov 22 '20 01:11 marius10p

So to replicate this problem, I should just set connected(n) = 0 for some channel n on NP 1.0 ?

marius10p avatar Nov 22 '20 01:11 marius10p

@marius10p Ah, I see. But why does KS2 find 385 channels on the probe when running the analysis then? Also, I was trying to find the connected variable in the rez.mat, but it was not there?

Idavr avatar Nov 22 '20 07:11 Idavr

I am also getting the exact same error and it appears to occur regardless if connected(:) = 1 or connected(anyN)=0.

Steps to reproduce

  1. Installed software: Windows 10, Open Ephys 0.5.2, Kilosort 2.5, phy2.0b1
  2. Open Ephys: record a session of FPGA data stored with recording note in binary format with a small data set (210 sec) of 32 channels + 8ADC (40 channels).
  3. Build channel map with connected(:)=1; OR connected(1:8)=0; % the other 32 channels being =1, save into .mat file
  4. Open KS2.5 GUI; select continuous.dat binary file recorded in Step 2 and channel map created in Step 3.
  5. Run All.
  6. In the output directory: phy template-gui params.py

End of my stack appears the same as the OPs:

  File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels
    assert best_channel in channel_ids
AssertionError
[0m

Wonder if @rossant was able to examine the data set of @ladvr and offer feedback?

Thanks!

linussun avatar Nov 23 '20 17:11 linussun

@Idavr I'm getting 383 channels in params.py, as I should (since I have channel 192 disconnected). What's strange is that that's the same setup that's used in neuropixPhase3B1_kilosortChanMap.mat and neuropixPhase3B2_kilosortChanMap.mat, so in theory if @marius10p has used those channel maps with KS2.5 he should be getting the same errors with Phy that we are, correct?

Hankslab-ABG avatar Nov 23 '20 22:11 Hankslab-ABG

Oddly, though, I'm not able to reproduce this issue with Phy for every recording. I just ran Phy successfully on a recording I did on Friday that has channel 192 disconnected, so I'm wondering if its actually related to the channel connections.

Also, for what it's worth, that channel does contribute to sorted clusters, so I'm not sure why that would be happening.

Hankslab-ABG avatar Nov 23 '20 22:11 Hankslab-ABG

@Hankslab-ABG, does it happen with some datasets 100% of the time and other datasets not at all? Is there a difference between the size of those datasets? I am wondering if the datasets are too small (in terms of time - as my test data sets are) that the error will occur more often.

Oddly, though, I'm not able to reproduce this issue with Phy for every recording. I just ran Phy successfully on a recording I did on Friday that has channel 192 disconnected, so I'm wondering if its actually related to the channel connections.

Also, for what it's worth, that channel does contribute to sorted clusters, so I'm not sure why that would be happening.

linussun avatar Nov 24 '20 02:11 linussun

Oddly enough I wasn't able to reproduce the issue on a dataset I've been sent. Could someone else send me a full dataset (raw data, .npy files, no need for the .phy subfolder) via dropbox or similar at my email address?

rossant avatar Nov 24 '20 09:11 rossant

@linussun , I'm still trying to suss out a pattern, but it seems like it happens more often for one of my rats than the other. They have a slightly different channel mapping as one rat has the NPx probe implanted slightly deeper than the other. Both run sessions that go from about 45 to 60 minutes, so they're not particularly short.

For what it's worth, here is the exact errors that are being thrown when I try to open Phy:

AssertionError [0m [90m12:36:58.872 [D] context:100 Initialize joblib cache dir at D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0.phy`.[0m

[90m12:36:58.872 [D] context:100 Initialize joblib cache dir at D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy.[0m

[90m12:36:58.872 [D] context:101 Reducing the size of the cache if needed.[0m

[90m12:36:58.872 [D] context:101 Reducing the size of the cache if needed.[0m

[90m12:36:58.874 [D] base:102 Add filter high_pass.[0m

[90m12:36:58.874 [D] base:102 Add filter high_pass.[0m

[90m12:36:58.874 [D] config:31 Load config file C:\Users\cns-hankslab\.phy\phy_config.py.[0m

[90m12:36:58.874 [D] config:31 Load config file C:\Users\cns-hankslab\.phy\phy_config.py.[0m [90m12:36:58.875 [D] plugin:145 Loading 0 plugins.[0m

[90m12:36:58.875 [D] plugin:145 Loading 0 plugins.[0m

[90m12:36:58.877 [D] context:209 The file D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\new_cluster_id.pkl doesn't exist.[0m

[90m12:36:58.877 [D] context:209 The file D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\new_cluster_id.pkl doesn't exist.[0m

[90m12:36:58.985 [D] context:185 Save data to D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\spikes_per_cluster.pkl.[0m

[90m12:36:58.985 [D] context:185 Save data to D:\Recordings\Rat78_2020-11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\.phy\spikes_per_cluster.pkl.[0m

[90m12:36:59.043 [D] gui:463 Creating GUI.[0m

[90m12:36:59.043 [D] gui:463 Creating GUI.[0m

[90m12:36:59.047 [D] state:46 Load C:\Users\cns-hankslab.phy\TemplateGUI\state.json for GUIState.[0m

[90m12:36:59.047 [D] state:46 Load C:\Users\cns-hankslab.phy\TemplateGUI\state.json for GUIState.[0m

[31m12:37:04.782 [E] init:62 An error has occurred (AssertionError): Traceback (most recent call last): File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec)

File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code exec(code, run_globals)

File "C:\ProgramData\Anaconda3\envs\phy2\Scripts\phy.exe_main_.py", line 7, in sys.exit(phycli())

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in call return self.main(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps_init_.py", line 159, in cli_template_gui template_gui(params_path, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui gui = controller.create_gui()

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui self.supervisor.attach(gui)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views gui, data=self.cluster_info, columns=self.columns, sort=sort)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info out[key] = func(cluster_id)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel channel_ids = self.get_best_channels(cluster_id)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels template = self.model.get_template(template_id)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template return self._get_template_dense(template_id, channel_ids=channel_ids)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in get_template_dense channel_ids, amplitude, best_channel = self._find_best_channels(template)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels assert best_channel in channel_ids

AssertionError [0m [31m12:37:04.782 [E] init:62 An error has occurred (AssertionError): Traceback (most recent call last): File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 193, in _run_module_as_main "main", mod_spec)

File "c:\programdata\anaconda3\envs\phy2\lib\runpy.py", line 85, in _run_code exec(code, run_globals)

File "C:\ProgramData\Anaconda3\envs\phy2\Scripts\phy.exe_main_.py", line 7, in sys.exit(phycli())

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 829, in call return self.main(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 782, in main rv = self.invoke(ctx)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1259, in invoke return _process_result(sub_ctx.command.invoke(sub_ctx))

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 1066, in invoke return ctx.invoke(self.callback, **ctx.params)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\core.py", line 610, in invoke return callback(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\click\decorators.py", line 21, in new_func return f(get_current_context(), *args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps_init_.py", line 159, in cli_template_gui template_gui(params_path, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 199, in template_gui gui = controller.create_gui()

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1638, in create_gui self.supervisor.attach(gui)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 942, in attach gui=gui, sort=gui.state.get('ClusterView', {}).get('current_sort', None))

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 760, in _create_views gui, data=self.cluster_info, columns=self.columns, sort=sort)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in cluster_info return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 916, in return [self.get_cluster_info(cluster_id) for cluster_id in self.clustering.cluster_ids]

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\cluster\supervisor.py", line 745, in get_cluster_info out[key] = func(cluster_id)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1150, in get_best_channel_label return self._get_channel_labels([self.get_best_channel(cluster_id)])[0]

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\base.py", line 1144, in get_best_channel channel_ids = self.get_best_channels(cluster_id)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\utils\context.py", line 154, in memcached out = f(*args, **kwargs)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phy\apps\template\gui.py", line 149, in get_best_channels template = self.model.get_template(template_id)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 855, in get_template return self._get_template_dense(template_id, channel_ids=channel_ids)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 798, in get_template_dense channel_ids, amplitude, best_channel = self._find_best_channels(template)

File "c:\programdata\anaconda3\envs\phy2\lib\site-packages\phylib\io\model.py", line 787, in _find_best_channels assert best_channel in channel_ids AssertionError [0m`

Also, two lines that appears in the cmd window but not in the Phy log is

`model:54 2928/135424 values are nan in D:\Recordings\Rat78_2020_11-23_11-40-47\experiment1\recording1\continuous\Neuropix-PXI-100.0\similar_templates.npy, replacing by zero.

c:\programdata\anaconda3\envs\Phy2\lib\site-packages\phylib\io\model.py:771: RunetimeWarning: invalid value encountered in greater_equal peak_channels = np.nonzero(amplitude >= self.amplitude_threshold * max_amp)`

Hankslab-ABG avatar Nov 24 '20 20:11 Hankslab-ABG

@rossant would a Google Drive link work for you?

Hankslab-ABG avatar Nov 24 '20 21:11 Hankslab-ABG

sure!

rossant avatar Nov 24 '20 21:11 rossant

@rossant Sent!

Hankslab-ABG avatar Nov 24 '20 22:11 Hankslab-ABG

Hi, Had the same problem in the past days, I just realised that a dataset will raise Assertion error when kilosort runs with minFR = 0 but not if minFR = 1/100. It seems like minFR has to be different than 0. @rossant I send you the datasets.

RobertoDF avatar Nov 28 '20 12:11 RobertoDF

Hi, I am having the same problem in multiple files. The files are about 30-40min long and recorded with NPX 1.0. I have tried some of the suggestions included in the issue without success:

  • setting minFR to 1/100
  • setting all channels to connected
  • using the default channelmap included in the Kilosort 2.5 repo wherein 192 is disconnected.

Let me know if sending the datasets might be useful. Thanks!

calbergaria avatar Nov 29 '20 20:11 calbergaria

Any updates on this?

For what it's worth, I re-ran a couple of the datasets that wouldn't open in Phy when using KS2.5. I used KS2 instead of 2.5 and was able to open them in Phy. However, what I found was that all of the clusters were based around very ill-defined waveforms that did not look spike-like at all. Has anyone else tried this? I know that some had mentioned noisy datasets might be behind this issue, so I'd be interested to see if anyone was able to get sensible-looking waveforms when re-running in KS2.

Hankslab-ABG avatar Dec 03 '20 19:12 Hankslab-ABG

What works for me now in Kilosort2.5:

  • setting ops.minFR = 1/600;
  • setting ops.nblocks = 0;
  • no removing any channel in the probe (all connected)

JoseGuzman avatar Dec 04 '20 14:12 JoseGuzman

I am experiencing this issue as well. I tried the workaround suggested by @JoseGuzman without success. @rossant, is there any update? I am also willing to share the recording I am using via dropbox if that would be helpful.

jbhunt avatar Dec 13 '20 20:12 jbhunt