core-bioimage-io-python
core-bioimage-io-python copied to clipboard
bioimage-io
Workflow RDF
needs https://github.com/bioimage-io/spec-bioimage-io/pull/478
Implements resolved nodes for the workflow RDF.
What do you think?
not sure if this really provides value. For debugging one can simply add a breakpoint in the for loop of the run_workflow op... Not sure it's a good idea to basically recommend debugging with print... If we want this we can of course add this, but I wouldn't conside thist urgent, as we are not sure how to proceed from this draft yet.
Well, think about a user use our package without access to the source code (which is the case for running in triton server).
This is not about debugging with print, you can use pdb the same way. Also, keep in mind that the run_workflow is inside our package. Vscode for example cannot add breakpoint for pip packages, but it can add breakpoint to user's own code. Even if we can add breakpoints for packages, not everyone uses an IDE. We don't want to ask everyone to clone our repo and tap into our source code, just for debugging a workflow.
For triton inference server, we pack the core into a conda package, and it runs inside a docker container, no pdb, no ide, print is the only way to debug. I would say this would be an important feature practically. Especially in the beginning, we want to know what happens when running the workflow.
Having the for loop in users hand can also allow easy UI integration, e.g. showing progress, customize error message for the failed steps. Even more powerful part would be change how we run the workflow steps, for example, I can replace the model execution step and send it to a triton model, or another triton server depending on the current workload.
It's a bit similar to how we train a model. We can of course use callbacks+run_workflow just like the Keras model.fit+callbacks, but often having pytorch-like user training loop provides more flexibility for debugging and usage.
You raise some good points I didn't think of yesterday. I'll look into implementing the workflow steps with yield... 👍
most content---workflow implementations and workflow exectuion logic--was moved to https://github.com/bioimage-io/workflows-bioimage-io-python the resolved nodes remain.
unfortunately I mixed in the renaming of ImportedSource to ImportedCallable into this PR and it's counterpart in spec: https://github.com/bioimage-io/spec-bioimage-io/pull/478
It's just some renaming to avoid confusion about source as in source data and source as in a python callable defined in a module or provided source code.
together with new repo https://github.com/bioimage-io/workflows-bioimage-io-python this updated PR supersedes #312 and #316