ipywidgets
ipywidgets copied to clipboard
ENH: Image should support rgb/rgba data
To be able to show 'raw' numpy arrays, we could support rgb and rgba format. I propose that the format trait support 'rgb' and 'rgba', which assumed the value contains rgb or rgba data in regular format (height, width, channel).
From the frontend side, this requires converting the rgb(a) data to for instance png encoded data. This can be done with a non-attached canvas element. Also, if we add two methods to ImageModel:
- setDataFrom(data, format)
- getDataAs(format) which would convert from to the supported formats to Image's own format type, we could support for instance the following use case:
- create a ipywebrtc.CameraStream that has an image attached (API under discussion with @martinRenou)
- take a snapshot (by sending an event), which can be grabbed as png encoded data at the frontend
- the CameraStream widget will call imagemodel.setDataFrom(pngdata, 'png)
- If the Image widget's format was set to 'rgb', the backend would be able to grab the image data and put it into a numpy array for analysis.
Feedback would be appriciated, I discussed it with @SylvainCorlay and he seemed to like it. If we get green light I may have found a new contributor.
This would be extremely helpful to me!