Feature: run in Browser with WebGPU

[Shchvova]

My attempt to build it for HTML5 with WASM using. Currently it builds but does not run:

cargo run-wasm blackjack_nodes

[setzer22]

Hey! Thanks for working on this :)

I can reproduce your steps so far, the code builds on my machine, but when I run it I get a runtime panic.

I got past the initial panic, which was due to the usage of the pollster to block on a future. The crate requires threads and that's not available on wasm, so I did a few adaptations to make it work using wasm-bindgen-futures instead.

Problem is, even after that I'm getting some other panic. There is this error on my console:

BrowserWebGpu Adapter 0: Err(
    LowDeviceLimit {
        ty: MaxVertexBufferArrayStride,
        device_limit: 0,
        required_limit: 128,
    },
)

You may have better luck on your end :thinking: because this looks related to driver/browser support, so different GPU / OS / browser combination may help there.

Since I can't push changes to your branch, here's a patch instead. You can save this onto a wasm_support.patch file at the repository root and apply it with git apply wasm_support.patch

diff --git a/Cargo.toml b/Cargo.toml
index b37465c..7d1f5dd 100644
--- a/Cargo.toml
+++ b/Cargo.toml
@@ -59,6 +59,9 @@ env_logger = { version = "0.9", default-features = false, features = ["termcolor
 profiling = "1.0"
 log = "0.4"
 console_log = "0.2"
+wasm-bindgen-futures = "0.4"
+
+
 
 
 [patch.crates-io]
diff --git a/src/app_window.rs b/src/app_window.rs
index 60efa56..65cc6a0 100644
--- a/src/app_window.rs
+++ b/src/app_window.rs
@@ -20,7 +20,7 @@ pub struct AppWindow {
 }
 
 impl AppWindow {
-    pub fn new() -> (Self, EventLoop<()>) {
+    pub async fn new() -> (Self, EventLoop<()>) {
         let event_loop = winit::event_loop::EventLoop::new();
         let window = {
             let mut builder = winit::window::WindowBuilder::new();
@@ -30,7 +30,7 @@ impl AppWindow {
 
         let window_size = window.inner_size();
         let scale_factor = window.scale_factor();
-        let render_ctx = RenderContext::new(&window);
+        let render_ctx = RenderContext::new(&window).await;
         let root_viewport = RootViewport::new(
             &render_ctx.renderer,
             UVec2::new(window_size.width, window_size.height),
diff --git a/src/main.rs b/src/main.rs
index dcfc189..8d87e27 100644
--- a/src/main.rs
+++ b/src/main.rs
@@ -28,13 +28,26 @@ pub mod math;
 /// General utility methods and helper traits
 pub mod utils;
 
-fn main() {
+async fn async_main() {
     // Setup logging
     #[cfg(not(target_arch = "wasm32"))]
     env_logger::init();
     #[cfg(target_arch = "wasm32")]
     console_log::init_with_level(log::Level::Debug).unwrap();
 
-    let (app_window, event_loop) = app_window::AppWindow::new();
+    let (app_window, event_loop) = app_window::AppWindow::new().await;
     app_window.run_app(event_loop);
 }
+
+fn main() {
+    #[cfg(target_arch = "wasm32")]
+    {
+        wasm_bindgen_futures::spawn_local(async_main());
+    }
+
+
+    #[cfg(not(target_arch = "wasm32"))]
+    {
+        pollster::block_on(async_main());
+    }
+}
diff --git a/src/render_context.rs b/src/render_context.rs
index cf6d029..951096f 100644
--- a/src/render_context.rs
+++ b/src/render_context.rs
@@ -3,6 +3,7 @@ use std::sync::Arc;
 use crate::{prelude::*, rendergraph::grid_routine::GridRoutine};
 
 use glam::Mat4;
+use rend3::RendererMode;
 use rend3_routine::pbr::PbrRoutine;
 use wgpu::{Features, Surface, TextureFormat};
 
@@ -21,14 +22,15 @@ pub struct RenderContext {
 }
 
 impl RenderContext {
-    pub fn new(window: &winit::window::Window) -> Self {
+    pub async fn new(window: &winit::window::Window) -> Self {
         let window_size = window.inner_size();
-        let iad = pollster::block_on(rend3::create_iad(
-            None,
+        let iad = rend3::create_iad(
             None,
             None,
+            Some(RendererMode::CpuPowered),
             Some(Features::POLYGON_MODE_LINE),
-        ))
+        )
+        .await
         .unwrap();
 
         let surface = Arc::new(unsafe { iad.instance.create_surface(&window) });

[Shchvova]

OK. I figured it out. I wasn't a mutex issue, it just internals of rend3 crashing on this call: https://github.com/BVE-Reborn/rend3/blob/d2a1717e799ab0a4eafaf8b4d9db8dc296146da1/rend3/src/setup.rs#L437-L444 (mutex was in error because it was panicking inside of pollster block)

[setzer22]

OK. I figured it out. I wasn't a mutex issue, it just internals of rend3 crashing on this call: https://github.com/BVE-Reborn/rend3/blob/d2a1717e799ab0a4eafaf8b4d9db8dc296146da1/rend3/src/setup.rs#L437-L444 (mutex was in error because it was panicking inside of pollster block)

Yup! pollster panicking on wasm is pretty much expected. See my comment above for a solution :+1:

[Shchvova]

Weird. You should be able to add origin https://github.com/Shchvova/blackjack.git and be able to push to the wasmRun branch. However my log debugging shows that it enters pollster closure but fails in the middle of it. May be it's because of .await, but I assumed it was earlier.

[setzer22]

I'm actually not sure where it fails exactly, but what I know for sure is that pollster is not designed to run on wasm, so the patch is required :smile:

I've tried to run this on windows under chrome canary. Also no luck :|

[setzer22]

Oh, yeah, I just realized I was able to push :sweat_smile: I wrongly assumed I wouldn't have permissions to push on your fork's branch :+1: Changes should be up now

[Shchvova]

When I run it in mainline browser it obviously fails. But it does progress further in the canary chrome (or firefox) with WebGPU in about://flags enabled. So, little progression. It now crashes on creating the surface:

        log::debug!("Windows Size is {:?} and iad {:?}", window_size, &iad.instance);
        let surface = Arc::new(unsafe { iad.instance.create_surface(&window) });
        log::debug!("Surface created");

Produces this output to console:

BrowserWebGpu Adapter 0: Ok(
    ExtendedAdapterInfo {
        name: "",
        vendor: Unknown(
            0,
        ),
        device: 0,
        device_type: Other,
        backend: BrowserWebGpu,
    },
)

Adapter usable in CpuPowered mode

Chosen adapter: ExtendedAdapterInfo {
    name: "",
    vendor: Unknown(
        0,
    ),
    device: 0,
    device_type: Other,
    backend: BrowserWebGpu,
}

Chosen backend: BrowserWebGpu

Chosen features: (empty)

Chosen limits: Limits {
    max_texture_dimension_1d: 8192,
    max_texture_dimension_2d: 8192,
    max_texture_dimension_3d: 2048,
    max_texture_array_layers: 256,
    max_bind_groups: 4,
    max_dynamic_uniform_buffers_per_pipeline_layout: 8,
    max_dynamic_storage_buffers_per_pipeline_layout: 4,
    max_sampled_textures_per_shader_stage: 16,
    max_samplers_per_shader_stage: 16,
    max_storage_buffers_per_shader_stage: 8,
    max_storage_textures_per_shader_stage: 4,
    max_uniform_buffers_per_shader_stage: 12,
    max_uniform_buffer_binding_size: 65536,
    max_storage_buffer_binding_size: 4294967295,
    max_vertex_buffers: 8,
    max_vertex_attributes: 16,
    max_vertex_buffer_array_stride: 2048,
    max_push_constant_size: 0,
    min_uniform_buffer_offset_alignment: 256,
    min_storage_buffer_offset_alignment: 256,
    max_inter_stage_shader_components: 60,
    max_compute_workgroup_storage_size: 16352,
    max_compute_invocations_per_workgroup: 256,
    max_compute_workgroup_size_x: 256,
    max_compute_workgroup_size_y: 256,
    max_compute_workgroup_size_z: 64,
    max_compute_workgroups_per_dimension: 65535,
}

Chosen mode: CpuPowered

Windows Size is PhysicalSize { width: 1024, height: 768 } and iad Instance { context: Context { type: "Web" } }

blackjack_nodes_bg.wasm:0x97e069 Uncaught (in promise) RuntimeError: unreachable
    at __rust_start_panic (blackjack_nodes_bg.wasm:0x97e069)
    at rust_panic (blackjack_nodes_bg.wasm:0x96519e)
    at std::panicking::rust_panic_with_hook::h47a0e203360b6c10 (blackjack_nodes_bg.wasm:0x67fb3c)
    at std::panicking::begin_panic_handler::{{closure}}::h888144d5e9a03cec (blackjack_nodes_bg.wasm:0x78d36f)
    at std::sys_common::backtrace::__rust_end_short_backtrace::h81961607fd97e28e (blackjack_nodes_bg.wasm:0x97c4be)
    at rust_begin_unwind (blackjack_nodes_bg.wasm:0x92c17d)
    at core::panicking::panic_fmt::h4ca53049f45e3264 (blackjack_nodes_bg.wasm:0x9557df)
    at core::panicking::panic_display::h5bb4c3c669911f66 (blackjack_nodes_bg.wasm:0x8ef2f3)
    at core::option::expect_failed::hdf65ec749c93533c (blackjack_nodes_bg.wasm:0x9651ee)
    at core::option::Option<T>::expect::h21558914aeddbca0 (blackjack_nodes_bg.wasm:0x7fc5b9)

Good news, it does receive adapter. Bad news, it crashes while creating surface. Adapter seems valid.

[Shchvova]

Great breakthrough 🔥 I realized that it was window error because we did not attach canvas to the body. Now it runs waaay past the starting point and fails on some silly stuff, and seemingly does a lot of some shader stuff and computing. Great progress :) Now it fails with this exception:

Uncaught (in promise) RuntimeError: unreachable
    at __rust_start_panic (blackjack_nodes_bg.wasm:0x97dfa6)
    at rust_panic (blackjack_nodes_bg.wasm:0x9650f5)
    at std::panicking::rust_panic_with_hook::h47a0e203360b6c10 (blackjack_nodes_bg.wasm:0x67f96a)
    at std::panicking::begin_panic_handler::{{closure}}::h888144d5e9a03cec (blackjack_nodes_bg.wasm:0x78d1fb)
    at std::sys_common::backtrace::__rust_end_short_backtrace::h81961607fd97e28e (blackjack_nodes_bg.wasm:0x97c3fb)
    at rust_begin_unwind (blackjack_nodes_bg.wasm:0x92c0d7)
    at core::panicking::panic_fmt::h4ca53049f45e3264 (blackjack_nodes_bg.wasm:0x955736)
    at core::result::unwrap_failed::h6d050dbd00b66340 (blackjack_nodes_bg.wasm:0x7b8e52)
    at core::result::Result<T,E>::expect::h6bd70f50383cbf14 (blackjack_nodes_bg.wasm:0x736d7e)
    at blackjack_nodes::mesh::debug_viz::load_obj_mesh::hc1b17c736ff8142b (blackjack_nodes_bg.wasm:0x38e0f1)

Full console output seems like some legit work being done localhost-1644858562732.log

It seems that loading some basic meshes fails because, you know, no file system in browser.

        let cylinder = renderer.add_mesh(load_obj_mesh("./assets/debug/arrow.obj"));
        let sphere = renderer.add_mesh(load_obj_mesh("./assets/debug/icosphere.obj"));
   -->
    let mut reader = BufReader::new(File::open(path).expect("File at path"));

I think it's a day for me.

[Shchvova]

OK... So I just hacked & slashed debug meshes (deleted all 3 lines containing debug_meshes) and Got this: Yay! Now it's a certainly day for me :)

[setzer22]

Amazing!! 🎉 I'm really excited about this

I also gave this a go, but couldn't figure out why the surface wasn't getting created so I gave up. I'm glad you didn't!

[Shchvova]

Today's progress was nice. I got UI working, and it is even responsive. However, main area doesn't seem to be working at all... Anyway. Last two commits contain some code I am not sure about at all. To load debug assets I embedded them into the binary with include_dir macro. However, that directory contains many other assets, so may be I should exclude them, or make a build script which would only embed necessary assets. Anyhow. I should note that I am extremely unexperienced in Rust, and not sure if my code is anyhow OK. Please, feel free to rewrite or ditch it.

[Shchvova]

I am stuck at this point. It seems that egui part of everything works just fine but for some reason main windows is not getting rendered. BTW, I run it in the trunk build of Chromium, downloaded from here: chrome-win.zip (link from here ) It spews bunch of warnings into console and seemingly no errors anymore:

Attachment state of [RenderPipeline "egui_pipeline"] is not compatible with the attachment state of [RenderPassEncoder]
 - While encoding [RenderPassEncoder].SetPipeline([RenderPipeline "egui_pipeline"]).
 ...
[Invalid CommandBuffer] is invalid.
    at ValidateObject (../../third_party/dawn/src/dawn/native/Device.cpp:564)

I am not sure what to do about this, or if they even matter.

[setzer22]

Once again, awesome work! :)

I've been a bit busy, but I want to take a more careful look at this later this week (probably tomorrow).

load debug assets I embedded them into the binary with include_dir macro

Your solution with include_dir sounds quite reasonable :+1: I am going to replace the code that draws vertices and edges with something better soon, so these debug assets may not be needed in the long term, but any small assets that are part of blackjack's "core" should be bundled in the binary to make it more portable and self-contained.

Please, feel free to rewrite or ditch it.

I've been having a quick look over your changes and so far everything looks quite nice :smile: I am certainly not ditching it!

for some reason main windows is not getting rendered

Huh.. :thinking: That's weird. I can imagine a few reasons for the top-left panel not working, but the bottom panel is using almost the exact same rendering code as the rest of the UI. Could it be that the bottom panel is being rendered, but there's just not anything in it? Does it respond to mouse input? Try right clicking somewhere inside the bottom panel, for instance. If nothing shows, also maybe try adding the following code at the start of draw_graph_editor, in graph_editor_egui.rs:16:

ctx.debug_painter().circle_filled(pos2(100.0, 100.0), 30.0, Color32::RED);

This should draw a red circle in the bottom panel. If you can see the circle, it means at least the bottom panel is working:

[Shchvova]

Thanks for the tips. No, it doesn't do rendering at all... Console shows a lot errors (warnings?) as I wrote above. I tried adding code, but it doesn't do anything. I also tried to add log to on_button_event in input.rs and it seem to get all the mouse move/click/scroll events. It seems that something is broken with rendering. May be it is something I did, since it was crashing on sleep, I deleted it in b17f90e52934dbc5470eb9d3eac1ed07511a350c. I don't know how to properly replace it with requestAnimationFrame or something similar.

[setzer22]

Removing the sleep code should work, it's only there to enforce we're rendering at a consistent framerate. That change would only make it so it runs as fast as your GPU can handle, but definitely not break rendering :thinking:

[Shchvova]

I tried to implement requesting animation frame, but honestly, at this point I have no idea how to combile event loop with callback nature of the JS.... I tried naive apporach, just waiting for callback to come, but obviously it didin't work since, no mutexes or multithreading on WASM. My apporach was inside the loop to request the animation frame, then use mutex and conditional variable to wait for the callback to happen, then to proceed with rendering. Well, it was never going to work :D

    #[cfg(target_arch = "wasm32")]
    fn on_main_events_cleared(&mut self) {
        use wasm_bindgen::closure::Closure;
        use wasm_bindgen::JsCast;
        use std::sync::{Arc, Mutex, Condvar};
        let pair = Arc::new((Mutex::new(false), Condvar::new()));
        let pair2 = Arc::clone(&pair);

        let closure = Closure::wrap(Box::new( move || {
            let (lock, cvar) = &*pair2;
            let mut started = lock.lock().unwrap();
            *started = true;
            cvar.notify_one();
        }) as Box<dyn FnMut()>);

        web_sys::window().unwrap().request_animation_frame( closure.as_ref().unchecked_ref()).unwrap();
        closure.forget();
        
        let (lock, cvar) = &*pair;
        let mut started = lock.lock().unwrap();
        while !*started {
            started = cvar.wait(started).unwrap();
        }
        
        self.root_viewport.update(&mut self.render_ctx);
        self.root_viewport.render(&mut self.render_ctx);

    }

I really know why I did it. But I think that in the end, WASM's on_main_events_cleared would be empty, but somewhere we will set up a request_animation_frame which will do update/render and then set up another request_animation_frame.

Then I tried to slow down rendering loop, and indeed, this errors are generated on every update/render cycle:

    #[cfg(target_arch = "wasm32")]
    fn on_main_events_cleared(&mut self) {
        if self.f%500 == 0 {
            log::info!("Loop {}", self.f);
            self.root_viewport.update(&mut self.render_ctx);
            self.root_viewport.render(&mut self.render_ctx);
        }
        self.f += 1;
    }

I think this is some good progress, but at this point I am stuck. This is very exciting project, but honestly, I have absolutely no idea what I am doing

P.S. I put built version here: https://svoka.com/blackjack_nodes/, it works only in Chromium, (not Firefox).

[setzer22]

Many thanks for all this amazing work! :smile:

Unfortunately, I tried running this and it doesn't seem to launch in any of the browsers I can install on my machine (tried all combinations of chromium / google chrome with stable, beta and dev). I'm getting some error related to browser limits I already shared above:

BrowserWebGpu Adapter 0: Err(
    LowDeviceLimit {
        ty: MaxVertexBufferArrayStride,
        device_limit: 0,
        required_limit: 128,
    },
)

Since I can't reproduce your results, it's a bit hard for me to debug the issue you're currently having with the rendering of the inner nodes. But seeing how you've come this far is quite reassuring! I'm confident we can get blackjack to run on browsers once webgpu support stabilizes a bit more :)

[setzer22]

Hey @Shchvova, just a heads up: The latest version (as of today) has made several changes in the way we use rend3 and wgpu. This means some of the roadblocks you encountered when trying to run this on the browser could have been improved:

• We now use the CpuDriven profile of rend3. This lessens the API requirements.
• Most of the 3d rendering code is made using custom wgpu draw calls now. I don't think I have used any technology that shouldn't be able to run in browser webgpu.
• All of the assets are now packed inside the binary via include_bytes!, so no more path errors.

Just in case you want to give this another try :) I can't promise it'll work this time, but we may see different results.