Structure of dash-canvas, integration with the dash ecosystem

[emmanuelle]

At the moment, dash-canvas is a crude wrapper around react-sketch. Its main feature is the ability to draw on an image and to retrieve the location of annotations for futher image processing operations. The javascript React component is minimal, exposing an image_content property for the background image and a json_data property for storing (among other things) the geometry of annotations.

Thanks to callbacks and python helper functions (exposed in image_processing_utils.py or plot_utils.py, pare_json.py etc.) it is possible to use annotations as

• markers for segmentation algorithms (as in app.py)
• starting from a labeled image (after a first segmentation), selection of objects to be merged, or objects to be separated in several objects (as in correct_segmentation.py)

These two parts (the minimal component and the set of python utils + apps) are in the same github repo for now. Should they be separated in two different python packages for more modularity? Could be dash-canvas and dash-canvas-utils. in the long term, the DashCanvas component could also be part of dash-core-components but I guess it's too early to decide.

I also wonder whether it would of interest to have dash-canvas as a trace of a dcc.Graph component? It could be interesting for example to re-use the mode bar (with the zoom in/ zoom out buttons, etc.).

[rpkyle]

I had the opportunity to discuss dash-canvas support in Dash for R with @dan-seol, who's been working on the image processing app. Some of his feedback:

• A native segmentation tool could be really useful; for both R and Python, a selection tool within dash-canvas would make it much easier to provide similar functionality across Dash backends
• A native implementation of the features currently provided by the draw module from scikit-image
• We could use the imager package to support selection (in my opinion, we'd ideally present the same interface across dash-canvas implementations)

Currently, there is no package which provides features analogous to those in scikit-image, though there exists a collection of package which each provide some of the features; I will try to identify which are not currently supported in an existing R package and summarize here. The packages include

• imager: https://cran.r-project.org/web/packages/imager/
• magick: https://cran.r-project.org/web/packages/magick/
• EBImage: https://www.bioconductor.org/packages/release/bioc/html/EBImage.html

In the meantime, I plan to check out this book, it looks fairly topical: https://www.springer.com/gp/book/9781447149491