pykilosort icon indicating copy to clipboard operation
pykilosort copied to clipboard

Set up benchmarking & testing against the current MATLAB version

Open alexmorley opened this issue 4 years ago • 3 comments

NB the results from this are likely to be hard to interpret without #18

Rough steps required:

  • [x] Run eMouse simulation in MATLAB
  • [x] Run Kilosort2 MATLAB version, serialize results to phy format
  • [x] Run Kilosort2 python version, serialize results to phy format
  • [ ] Determine metrics for similarity - which units found, how many spike times they share etc. IN PROGRESS
  • [ ] Script to automatically do all of the above automatically. IN PROGRESS
  • [ ] @rossant suggestion --> to find the divergence point in the implementation, compare the outputs of the different steps: preprocessing, main loop, postprocessing, when each step receives the same inputs in both MATLAB and Python IN PROGRESS
  • [ ] Test using the process from above but pulling the eMouse simulation file and the MATLAB results from the internet (or using locally saved files) - this test should be independent of MATLAB and will serve as a regression / parity test.

alexmorley avatar May 25 '20 18:05 alexmorley