python-neo
python-neo copied to clipboard
NeuralEnsemble
Feature/new filter
Until now it was not so easy to filter for multiple values with the filter function in neo. To solve this problem, I introduced "FilterConditions". These can be provided to the filter function as the value of a dictionary.
An example of the syntax:
seg.filter({“electrode_id”: FilterCondition) or seg.filter(electrode_id = FilterCondition)
In order to be able to filter for several electrode_ids, there are some objects that inherit from FilterCondition and provide different filtering options.
For example, seg.filter({"electrode_id": neo.FilterIsIn([0, 1, 5]) can be used to filter for all objects with electrode_id 0, 1 or 5.
List of all currently added FilterConditions:
-FilterEqual(x), returns id == x
-FilterIsNot(x), returns id != x
-FilterLessThan(x), returns id < x
-FilterLessThanEqual(x), returns id <= x
-FilterGreaterThan(x), returns id > x
-FilterGreaterThanEqual(x), returns id >= x
-FilterIsIn([x, y, z]), returns id == x or id == y or id == z
-FilterInRange(a, b), returns a <= id <= b
Additional FilterConditions can be added at any time and without much effort.
Hello @kloss-o! Thanks for updating this PR. We checked the lines you've touched for PEP 8 issues, and found:
There are currently no PEP 8 issues detected in this Pull Request. Cheers! :beers:
Comment last updated at 2022-06-28 12:08:16 UTC
Thanks for the PR, Oliver, and for reviewing the code, Julia.
Regarding performance, there is actually another follow-up PR by @kloss-o in prep that would improve filtering speed quite a bit, but unrelated to this concrete implementation. @kloss-o , I think when you did some benchmarking the FilterCondition implementation (without further optimization) was comparable in speed to the traditional implementation?
The idea behind the different Filter classes was, for one, that it would allow users to also define their own "custom" filters as a filter function, like (in pseudocode):
class FilterGoodElectrodesSurroundedByGoodOnes(FilterCondition)
def test(electrode):
return is_good_quality[electrode] and np.all(is_good_quality[i] for i in neighbor_electrodes(electrode])
Secondly, that it would not require any new or ambiguous syntax (e.g., something like ["property", "<=", 5]) in the function call to filter(). In particular, the neo filter function would continue to work as before with the existing syntax, but adds additional functionality.
However, I think there are many other options. An alternative could be to revert to, e.g., simple filter functions, at the expense of being slightly more ambiguous.
The following notebook is intended to show the application of the 'new filter' function in practice. The use cases were derived from the Neo - Elephant tutorials and aim to demonstrate how the suggested 'new filter' improves the workflow.
Link to notebook tutorial_new_filter_examples.ipynb:
GitHub gist: https://gist.github.com/Moritz-Alexander-Kern/afdfb707bde9052c6cd02ccddd2c4b9e GIN: https://gin.g-node.org/NeuralEnsemble/neo_elephant_teaching/src/new_filter/tutorial_new_filter_examples.ipynb
Hi @kloss-o @Moritz-Alexander-Kern any news on this? Tell me if the notes from the last meeting are not detailed enough for you to continue on this topic.
-I moved the filter classes to a separate module.
-I also managed to decrease the filtering time. For comparison, I created one Segment that contains x Spiketrains and use a filter that applies to every Spiketrain. (Tested on multiple combinations of Segments, Blocks, Spiketrains, Analogsignals, the result is nearly the same).
Old filtering method:
New filtering method:
Hey @apdavison and @JuliaSprenger , Thank you for the reviews and the valuable feedback you provided. I considered all your comments and made changes to the code.
If you have any further suggestions or if there are any remaining issues, please don't hesitate to let me or @mdenker know.