[ENH] `merge_index` method for TsGroup?

[qian-chu]

Possible user scenario

Take a TsGroup containing Ts of spikes indexed by neurons. Users might realize that some "neurons" are actually not distinct and belong to the same neuron (e.g. due to over-clustering during spike sorting). Instead of re-performing spike sorting and import data into Pynapple again, users might want to simply merge those indexes.

Proposed implementation

def merge_index(indices, reset_index=False)

Where indices is a list of indices or a list of list of indices. reset_index controls whether the TsGroup's indices are reset to 0, 1, 2, 3 ... If False, the smallest index is to be used as the index after merging.

Let me know what you think and happy to init a PR!

[BalzaniEdoardo]

Thanks for the suggestion, I like the idea. Have you thought of a way to merge the metadata? How would this merge behave when the two neurons have some metadata discrepancies?

[qian-chu]

Thanks for the suggestion, I like the idea. Have you thought of a way to merge the metadata? How would this merge behave when the two neurons have some metadata discrepancies?

Very good question... Simplist (but probably not the most desirable) solution would be to use the metadata of the smallest index. I can also imagine users passing on new metadata as pd.Series (merging a set of indices) or pd.DataFrame (sets of indices), but this sounds a bit dirty. What do you think?

btw, it seems that the docstring for some functions like https://pynapple.org/generated/pynapple.TsGroup.update.html needs some formatting to be correctly rendered by sphinx. If I ended up using .update() I might just fix that.

[BalzaniEdoardo]

It is a bit tricky, thinking about it more, pynapple might not be the right place for this type of processing. Let's hear what @gviejo thinks about this...

[gviejo]

I somehow agree with Edoardo. I don't see a good way to deal with metadata and merging units seems like something to be done before using pynapple. To me, it's a tool you use once you have stabilized your data. We can leave this issue open for the moment and if more and more people request it, we can eventually think of a way to implement it.