python-neo icon indicating copy to clipboard operation
python-neo copied to clipboard

Update NixRawIO to strictly follow RawIO data structure

Open JuliaSprenger opened this issue 2 years ago • 5 comments

This PR rewrites the IO to read only data that follow the RawIO restrictions. For loading old data autogenerate_* keywords were introduced to permit loading data, but ignoring inconsistent channel / unit names. Since there is no structural relation between blocks in nix, it is only possible to load a single block at time. The substructure of that block has to follow the rawio constraints.

This PR is requires test files added in https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/pulls/100 .

JuliaSprenger avatar Apr 04 '23 08:04 JuliaSprenger

@Kleinjohann Can you test this implementation with some of your nix files?

JuliaSprenger avatar Apr 11 '23 14:04 JuliaSprenger

With https://gin.g-node.org/NeuralEnsemble/ephy_testing_data/pulls/100 merged, tests are passing now.

JuliaSprenger avatar Apr 26 '23 13:04 JuliaSprenger

@Kleinjohann Can you test this implementation with some of your nix files?

Hi! Thanks for the work! I tested it with our files and it works. Initialising the IO and lazy-loading a block takes quite long, but everything seems to work.

Kleinjohann avatar Apr 27 '23 15:04 Kleinjohann

Todos:

  • [ ] Improve parse_header runtime

JuliaSprenger avatar Apr 27 '23 15:04 JuliaSprenger

@JuliaSprenger do you want to implement your "todo" item, or shall we merge now and fix that later?

apdavison avatar Jul 27 '23 16:07 apdavison