WebGPU rendering backend

[Gerharddc]

trafficstars

This project looks very interesting, but I am wondering if a WebGPU based rendering backend (probably using https://github.com/gfx-rs/wgpu-rs) would not be more appropriate for a modern use case than an OpenGL ES 2.0 one as this would allow for modern standards like Vulkan, Metal and DX12 to be supported automatically.

[tronical]

Yes, I think gfx-rs/wgpu-rs is awesome and it'll make sense in the future to use it. At the moment if we switched to gfx-rs we would loose the ability to run our demos in Safari on iOS/iPadOS/macOS because that's limited to WebGL 1. The ability to run our demos in the browser has been of great help, but I believe there's a good chance that upcoming versions of Safari will enable WebGL 2 out of the box.

We're also using the femtovg renderer now, on top of glow. So the effort of switching to gfx-rs will mean contributing to the effort of adding a gfx-rs backend to femtovg (which makes sense, no doubt).

[Gerharddc]

Interesting, I have never heard of femtovg (did see nanovg it is based on years ago though), but yeah it obviously makes sense to add it as a back-end there if you are using that for rendering. So if something like femotovg is used then I guess this means that sixtyfps uses a traditional imperative drawing approach instead of a Qt Quick style scene graph approach?

[tronical]

The approaches are rather similar:

The qtquick item tree creates a maintains a scene graph node tree at runtime, which is traversed to build GL commands (draw and minimal geometry and texture buffer).

SixtyFPS creates an item tree at compile time that’s equivalent to the scene graph node tree. It’s traversed and femtovg generates list of GL commands (draw, geometry buffers and minimal texture).

So qtquick has slightly better geometry batching (which has advantages but also it’s own caveats) but maintains two trees. SixtyFPS maintains a tree less, does more work ahead of time but femtovg doesn’t do geometry batching.

There might be holes in my understanding, so if I missed something please correct me :)

[Gerharddc]

I see. Well yes the main difference that I am thinking about is that in Qt Quick you cannot issue "drawing commands" in order to render elements, you have to manually construct "geometry" and "materials" which more closely maps to GL primitives and I understand that building up a tree of such things is what allows for the efficient batching.

And I guess the other advantage of defining elements on a more low level like that is that "redrawing" then does not have to be done from scratch. I presume that when using femtovg and you want to for instance change the background colour of a button, then a complete new set of GL commands are calculated and sent to the GPU, whereas in Qt Quick a component could directly inform the rendering engine that it only needs to change one uniform and does not need to replace the entire vertex buffer for instance.

I must admit that I have no idea if what I just described is actually what using femtovg entails, but I would image that it is very hard to make a library that accepts high level drawing commands, and then manages to directly calculate the delta in GL commands between two draw calls with slightly different parameters. Anything I can imagine to implement would simply calculate the GL commands from scratch at each drawing call and if I want to be fancy, I would store the old GL commands and then difference them with the current one in order to at least try and minimize communication with the GPU (I have no idea if anything actually does this).

But then again, forcing component creators to define things on such a low level is a horrible experience for them as opposed to the very pleasant experience of using high level drawing commands.

I know that Flutter manages to achieve very high performance while also using high level drawing calls (in their case directed at Skia), but I have not been able to figure out if there is some serious magic in their system or if repainting from scratch just isn't as bad as it sounds most of the time.

[tronical]

I've been thinking a bit more about this and I think we're close to a point where it makes sense to assign a higher priority to this. A few data points:

• Tickets like #423 will require the GL backend to expose API. Maybe it's better if this were wgpu based.
• All the wasm platforms we support for demos now also support WebGL 2 (GLES 3.0), which in turn wgpu supports as a backend.

[Be-ing]

A use case this would help with is running SixtyFPS applications in a Windows VM from Linux with QEMU-KVM. My SixtyFPS application builds on Windows, but it panics at runtime because it can't create an OpenGL context:

thread 'main' panicked at 'Failed to create OpenGL context: could not create GlLatest context : Couldn't find any pixel format that matches the criteria.', C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-rendering-backend-gl-0.1.6\glcontext.rs:163:17
stack backtrace:
   0: std::panicking::begin_panic_handler
             at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b\/library\std\src\panicking.rs:498
   1: core::panicking::panic_fmt
             at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b\/library\core\src\panicking.rs:107
   2: sixtyfps_rendering_backend_gl::glcontext::impl$0::new_context_and_renderer::closure$0
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-rendering-backend-gl-0.1.6\glcontext.rs:163
   3: sixtyfps_rendering_backend_gl::event_loop::with_window_target::closure$1
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-rendering-backend-gl-0.1.6\event_loop.rs:278
   4: std::thread::local::LocalKey<core::cell::RefCell<enum$<core::option::Option<sixtyfps_rendering_backend_gl::event_loop::NotRunningEventLoop>, 0, 3, Some> > >::try_with
             at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b\library\std\src\thread\local.rs:399
   5: std::thread::local::LocalKey<core::cell::RefCell<enum$<core::option::Option<sixtyfps_rendering_backend_gl::event_loop::NotRunningEventLoop>, 0, 3, Some> > >::with<core::cell::RefCell<enum$<core::option::Option<sixtyfps_rendering_backend_gl::event_loop::No
             at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b\library\std\src\thread\local.rs:379
   6: sixtyfps_rendering_backend_gl::event_loop::with_window_target<glutin::windowed::ContextWrapper<enum$<glutin::context::NotCurrent>,winit::window::Window>,sixtyfps_rendering_backend_gl::glcontext::impl$0::new_context_and_renderer::closure$0>
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-rendering-backend-gl-0.1.6\event_loop.rs:274
   7: sixtyfps_rendering_backend_gl::glcontext::OpenGLContext::new_context_and_renderer
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-rendering-backend-gl-0.1.6\glcontext.rs:104
   8: sixtyfps_rendering_backend_gl::glwindow::impl$2::show
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-rendering-backend-gl-0.1.6\glwindow.rs:378
   9: sixtyfps_corelib::window::Window::show
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-corelib-0.1.6\window.rs:421
  10: sixtyfps_corelib::window::api::Window::show
             at C:\Users\Be\.cargo\registry\src\github.com-1ecc6299db9ec823\sixtyfps-corelib-0.1.6\window.rs:586
  11: moire::gui::generated::sixtyfps_generated_MainWindow::impl$34::show
             at .\target\debug\build\moire-181ad4cf19ee6568\out\main_window.rs:4079
  12: moire::gui::generated::sixtyfps_generated_MainWindow::impl$34::run
             at .\target\debug\build\moire-181ad4cf19ee6568\out\main_window.rs:4074
  13: moire::gui::Gui::run
             at .\src\gui.rs:96
  14: moire::main
             at .\src\main.rs:32
  15: core::ops::function::FnOnce::call_once
             at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b\library\core\src\ops\function.rs:227
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
error: process didn't exit successfully: `target\debug\moire.exe` (exit code: 101)

I don't have any computers running Windows on bare metal so I'm currently unable to test my application on Windows. There was a GSOC project in 2017 to write a Windows driver for OpenGL within QEMU VMs, but that has not been completed. However, the wgpu examples run in this environment (albeit slowly):

[tgross35]

It seems like maybe femtovg has a chance of supporting this sometime soon (hopefully) https://github.com/femtovg/femtovg/issues/177

Could it be possible to get a wGPU canvas that we can use shaders on? It seems like this could be possible without support for a full rendering backend, just a rectangle area that gets redrawn after everything else

[jquesada2016]

Any updates on this?

[Horbin-Magician]

I'm curious, too. Any updates on this?

[tronical]

Unfortunately no. Would you have any recommendations for a wgpu based renderer that's on crates.io?

[tronical]

I see vello nowadays has updates and looks very promising.

[Horbin-Magician]

I see vello nowadays has updates and looks very promising.

I did some research, and agree that vello is the most appropriate library by far.

[xb284524239]

I noticed that the FemtoVG library has added support for WGPU in version 0.11.0. Does it mean that we can also choose to use the WGPU backend in Slint soon? Because the building speed based on the Skia backend is really, really slow (on my work laptop)....

[tronical]

That’s right:). I’ve got a work in progress branch for that for Slint and plan to make a PR. It won’t perform miracles ;-) but it’ll be an opt-in.

[amitsheokand]

That’s right:). I’ve got a work in progress branch for that for Slint and plan to make a PR. It won’t perform miracles ;-) but it’ll be an opt-in.

I wanted to test it out, but It is incomplete, slint::GraphicsAPI still shows NativeOpenGL and WebGL. we have a big application written in Electron and Rust but we want to port frontend to Slint. If It is on roadmap, we can also discuss commercial aspect of it. If you can give a timeline that would be great. Thank you for great work on Slint.

[tronical]

I'll rebase my branch on Monday. Do you need a GraphicsAPI / rendering notifier integration? If yes, what kind of functionality do you need?

In theory we could expose wgpu types there, but wgpu apis change, so those parts of the slint api would also not be stable.

Let us know what you need and then we can make a plan :)