Project-MONAI
[Feature] Support simple (as in functional API) visualisation of appropriate (grayscale or RGB image) GPU tensors without transitioning back to host
Is your feature request related to a problem? Please describe. It would be great to have an easy to use means of directly visualising a PyTorch GPU tensor that represents a standard grayscale of RGB image. For efficiency reasons, we would want to avoid trnsfering the data back to host.
Describe the solution you'd like
Something as simple to use as the OpenCV cv.imshow (see here and here) as shown in the tutorial:
https://docs.opencv.org/5.x/dd/d43/tutorial_py_video_display.html
Describe alternatives you've considered Transfering to host and use numpy to display using OpenCV
Additional context Snippets of relevant code can be found on the web for this task but it's not trivial to use: https://gist.github.com/victor-shepardson/5b3d3087dc2b4817b9bffdb8e87a57c4
In terms of visualisation, vulkan would be preferable on the long run but opengl may be good enough for most purposes.
Also, proper GUI work is probably out of scope for MONAIStream. That said providing a simple means of displaying the content of an appropriate PyTorch GPU tensor in any suitable GUI toolkit without going back to host would be a great step forward. MONAILabel did something similar with Slicer. For MONAI Stream, in terms of GUI toolkit, it's unclear to me what makes sense. Slicer is probably not the simplest as it's a full blown app rather than a toolkit. Something like Datoviz / Dear PyGui may be interesting to look at. I guess talking to the VTK devs would also be super helpful.
Some other relevant points:
- CUDA Python 11.6 exposes the OpenGL-CUDA interop APIs. https://nvidia.github.io/cuda-python/release/11.6.0-notes.html#cuda-python-11-6-0-release-notes
- A discussion on similar questions but for cupy and vispy can be found here: https://github.com/vispy/vispy/issues/1985
