ImJoy
ImJoy copied to clipboard
Redesign the dataloader
Currently, we use json schema to match data types, but suffering from performance loss during type checking like we try to fix here: https://github.com/oeway/ImJoy/pull/297 .
We need to revisit the design and try to figure out a more explicit way to doing type checking.
One potential solution is to deprecate the inputs and outputs in the data-loader functions to handle files or data produced by other plugins.
However, allowing custom data-loader would mean all the data will transfer to each plugin which provides the data-loader, which perhaps is not a good idea. We may need to either stick with the schema, or somehow have a safe way to execute the script.