ImJoy
ImJoy copied to clipboard
imjoy-team
Support test api for plugins
Since tests are important to verify and guarantee whether a certain plugin is functioning, or on the other hand check if changes in ImJoy breaks certain plugins. Right now there is no way to do it automatically, but this could be solved by introducing a test function to the plugin interface.
Here is what I propose:
- each plugin should (must?) define an api function called
test()along withsetup()in the plugin class. - the
test()function should accept no argument exceptselffor python, and it should bootstrap itself, generate/simulate/download data, run other functions defined in the plugin, verify the result and throw error (with error message). - add a test button on the code editor toolbar to allow developers quickly run this test function during developing.
- setup Travis to do automatic test all the plugins hosted on oeway/ImJoy-Plugins and oeway/ImJoy-Demo-Plugins with the latest version of ImJoy.
@MartinHjelmare I posted this issue before, and I think it would be nice to automate plugin test, ideally, after these mechanism is ready, we should be able to easily load plugins from an entire plugin repository (e.g.: ImJoy-Plugins) and click a button to test these plugins one by one, and generate a report on the fly, and this can also be integrated with the CI we have, we can basically run these test before deploy, we can then either fix the plugins or the app.
How about the plugin test function name, should we just use test and reserve it for special use or there is better alternative names for this, we can also do __test__ to make the name special and avoid potential naming conflicts.
For consistency with setup, resume and run, perhaps we can just use test.
Just discussed with @constantinpape @FynnBe and others, it would be nice to support a prefix test_ for the testing functions instead of use just test. I think it's a good idea, then we can have multiple tests in the plugin file.