BUG: KS2 fails with huge numbers of events and there is an easy fix

[brandonStell]

Describe the issue:

Adding a fallback to "economy" decomposition prevents KS2 from failing when detecting huge numbers of events:

        "Number of spikes before applying cutoff: 36449620",
        "----------------------------------------Error using gpuArray/eig",
        "EIG did not converge at index = 881672."

Adding the below fallback to runTemplates.m solves the issue. https://github.com/MouseLand/Kilosort/blob/v2.0.2/mainLoop/runTemplates.m#L91

    try
        [A, B, C] = svdecon(UA);
    catch
        [A, B, C] = svd(UA,"econ");
    end

The economy-size decomposition removes extra rows or columns of zeros from the diagonal matrix of singular values, S, along with the columns in either U or V that multiply those zeros in the expression A = USV'. Removing these zeros and columns can improve execution time and reduce storage requirements without compromising the accuracy of the decomposition. https://www.mathworks.com/help/matlab/ref/double.svd.html#d126e1780458

Reproduce the bug:

Error message:

        "Number of spikes before applying cutoff: 36449620",
        "----------------------------------------Error using gpuArray/eig",
        "EIG did not converge at index = 881672."

Version information:

This is on all versions.

[jacobpennington]

I'll point this out to @marius10p in case it's something he wants to add a patch for, but we are not providing support or development for previous versions of Kilosort anymore. It looks like you've figured out a fix for it for your own use. If you run into anything that prevents you from using a past version, we recommend trying Kilosort4 instead.

[brandonStell]

Thanks for getting back to me. We find it very useful to run a bunch of sorters and then merge the results using Lussac. The most reliable way we’ve found to run all sorters is via the Docker containers provided by SpikeInterface, which in turn depend on your repository to build and maintain those images.

At the moment, I’m unable to run Kilosort2 (one of our favorites) in the pipeline because I can’t access the Docker container to apply this simple modification manually. If the change were implemented directly in your repo, SpikeInterface could then update their Docker image accordingly, which would make it much easier to include KS2 in our workflow.

[jacobpennington]

I understand, and I made sure @marius10p is aware of the issue, but we are no longer pushing updates for past versions of Kilosort. I would recommend checking with the SpikeInterface team to see if they have suggestions for using your own fork of Kilosort2 with their Docker containers.