iced
iced copied to clipboard
iced-rs
Rendering to FBO
I'd like to render video using libmpv-rs inside an iced GUI. There are different ways to do it, but the best would be to render to an FBO. Is this possible with iced today (without bending it too much)?
This issue has been open for a long time. Realistically what's blocking this? And how difficult would it be to implement this?
Out of curiousity I decided to take a quick look at this.
libmpvhas an empty render module — this part of the API isn't wrapped yet (see the mpv render API)- MPV supports two types of embedding: native window embedding (OS-specific and not supported by winit), and the "render API", which supports OpenGL and software rendering
Of these, the software rendering looks fairly easy: just create a buffer in memory, tell MPV a few things about the format, and pass the pointer with MPV_RENDER_PARAM_SW_POINTER, then copy the contents to a texture. (Using a Canvas or Image widget for the upload maybe? Not sure how Iced supports this.) MPV notes that performance may not be sufficient for real-time video (depending on output size).
The OpenGL backend would obviously be preferable, but Iced doesn't even use OpenGL by default (except possibly through WGPU in the near future, but then its context would not be directly accessible). It's also beyond a "quick look".
@dhardy I have a fork where I worked on wrapping the render api. As for iced itself, it has glow backend, doesn't it? Or is that no more? I opened a PR way back to add an integration example. But there was no feedback so didn't really complete it. I expect it will be incompatible with the current iced.
@sztomi I'm not very familiar with Iced but just paying attention since https://github.com/kas-gui/kas/ has many of the same underlying libraries and issues. I believe Iced does have an (incomplete) glow backend; KAS does not. KAS does support embedded WGPU pipelines, but I'm not aware of any video renderer over WGPU so it doesn't really help.
@dhardy yeah, it seems that all upcoming Rust GUI libs opt for wgpu-first and neglect opengl. Which is a shame because even though wgpu will allow squeezing out more performance, opengl is more useful for all kinds of integrations, not to mention API simplicity. wgpu demands a lot more work and understanding from the API user. The glow backend did work for me when I ported the integration example of iced to it.
@sztomi it's still another API to learn, and I happened to familiarise myself with WGPU first. That said, Iced has backends for both (though I hear the glow backend is less complete), and there's no reason KAS couldn't also support both.
WGPU targets all modern APIs and now also has working OpenGL compatibility, so I don't think it's objectively worse — except that (a) it's still immature and (b) libraries like MPV which specifically require an OpenGL context are incompatible.
@dhardy FYI there is a PR open in libmpv-rs to wrap the rendering module that is close to completion: https://github.com/ParadoxSpiral/libmpv-rs/pull/22
That libmpv-rs PR works great, was able to display a video with fltk-rs. Iced, as of now, cannot do this because it doesn't support OpenGL? Am i understanding this correctly?
@ergpopler no, iced does support opengl via the glow backend. @hecrj kindly updated my PR for an example of opengl integration and it is merged now. You should be able to take the mpv-rs PR branch and render iced on top of it using that example as a starting point.
The important part of the libmpv API is that the only point where it has to connect to OpenGL is the glGetProcAddress function call. If you manage to forward the call and the response properly, you're golden (plus you need to specify the framebuffer size in pixels, which should be easy).
This makes it very easy to integrate, but limits you to OpenGL (no Vulkan, wgpu etc). If you really want to use another rendering API, you might be able to achieve this using ANGLE, but I have no experience with that.
