Support for custom napari widgets (i.e. movement GUI plugins)

[adamltyson]

Movement is primarily a Python API as that is effectively the only way to support every possible type of data loading/processing/analysis/visualisation/saving that we would like.

The GUI will likely support two types of processing:

• Interactive visual, i.e. the types of processing that it would be too cumbersome to do any other way (e.g. playing videos, refining ROIs)
• Very common tasks, i.e. a small subset of movement functionality that we think many (most) users, particularly novice users would like to do.

While it would be fantastic to make a GUI version of every single movement function, it would be a huge amount of work, lead to a chaotic UI, and be a huge amount of work to maintain.

Therefore, my bright idea (and feel free to tell me why it's stupid) is to support "movement GUI plugins". These would be additional napari widgets that allow the user to carry out a specific task in an entirely graphical environment. This could be a lab- or field-specific task. In principle they could be very lightweight, leveraging napari for general visualisation and the existing movement napari plugin for basic functionality such as I/O. I'm thinking things like:

• Loading non-standard formats
• Integrating with other packages (e.g. for aligning physiology and behaviour)
• Creating standard visualisations
• Carrying out routine, lab specific analyses

My plan is for these widgets to be primarily created by the community, the work needed on the movement side would be to support them by:

• Supporting the "link" between new plugins and the existing "standard" plugin
• Potentially updating the API to provide optional parameters relating to napari (not sure what these would be, and this may not be necessary)
• Developing a small number of examples

I think this has a lot of benefits:

• Movement can indirectly support a far wider array of GUI-based workflows
• Users can more easily build GUIs for their lab (this is primarily via educating users about napari plugins)
• The core dev team wouldn't need to provide maintenance for these lab-specific workflows
• The napari plugin infrastructure takes care of dissemination

We could of course "adopt" particularly good/useful plugins if everyone agrees and make them part of a supported "movement-extensions" package.

Interested in the thoughts of @neuroinformatics-unit/behaviour (and anyone else).

[sfmig]

I think this is a good idea.

The GUI will likely support two types of processing:

• Interactive visual, i.e. the types of processing that it would be too cumbersome to do any other way (e.g. playing videos, refining ROIs)
• Very common tasks, i.e. a small subset of movement functionality that we think many (most) users, particularly novice users would like to do.

I like this distinction, I think it will be very useful to help us prioritise things. Is the suggestion that we focus mostly on the functionality to support the first bulletpoint?

While it would be fantastic to make a GUI version of every single movement function, it would be a huge amount of work, lead to a chaotic UI, and be a huge amount of work to maintain.

I agree with this too and had been thinking about it for a while.

It would be nice if we could provide a "movement-plugin" template, and maybe worth it to have a go ourselves at one to get a feel of how it would look like.

[adamltyson]

It would be nice if we could provide a "movement-plugin" template, and maybe worth it to have a go ourselves at one to get a feel of how it would look like.

We may want to create a few, simple, different, plugins to get a feel for the "gotchas". @Sepidak's lab will also likely want to create something like this for spatial navigation analysis at some point.

[niksirbi]

This is a good idea and I agree with all of @sfmig's comments. So the movement-napari roadmap would be:

1. Focus on improving interactive visualisation capabilities (which we are already doing).
2. Focus on implementing necessary widgets in the core plugin, like drawing ROIs and tagging events
3. For other functionalities experiment with standalone napari plugins (that play nicely with our main plugin) and eventually come up with a movement-napari-plugin template.