Add real-time output streaming in headless browser mode.

[drewcrawford]

Description

Add real-time output streaming in headless browser mode.

Motivation

Suppose my code has a bug - a deadlock, an infinite loop, some browser-specific issue. My test hangs. The normal approach is printf debugging: add some web_sys statements, run again, see where it gets to.

In headless mode, when my test hangs, this doesn't work:

Waiting for test to finish...
# ... 60 seconds of silence ...
Failed to detect test as having been run. It might have timed out.

Output is buffered until completion. If the test never completes, I never see my prints. I don't know which test hung, where it hung, or whether my debug statements even executed.

With this PR, output from the main browser thread streams as it appears:

running 3 tests
test foo ... ok
test bar ... ok
test hanging_test ...

Now printf debugging works. I can add prints, run, and see where execution stops.

Checklist

• [x] Verified changelog requirement

[codspeed-hq[bot]]

CodSpeed Performance Report

Merging #4845 will not alter performance

_{Comparing drewcrawford:realtime-headless-output (84a3683) with main (315cfa1)}

Summary

✅ 4 untouched

[Spxg]

There is a text overlapping issue when running in run_in_shared_worker and run_in_dedicated_worker.

[guybedford]

There is a text overlapping issue when running in run_in_shared_worker and run_in_dedicated_worker.

Can you clarify what you mean by this? Is that a change required in this PR further or an issue with the approach?

[Spxg]

Is that a change required in this PR further

Yes, I apologize for not being clear, it's the test output issue in run_in_*_worker mode caused by this PR.

Running headless tests in Chrome on `http://127.0.0.1:41923/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
running 1 test                                    
test test_1 ... ok

=>

Running headless tests in Chrome on `http://127.0.0.1:35851/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
Loading Wasm module...st_1 ... ok  

[drewcrawford]

I think @Spxg found a legitimate regression. I have added what I think is test coverage for that issue to test_headless_worker_output_not_garbled so that my own understanding of that regression can be externally replicated. CI is telling me I found the failure, and I believe it.

Looking into the regression, I think it touches on several long-standing architectural issues in wasm-bindgen that have proven difficult to PR.

Accordingly

1. Making this PR a draft until I figure out how to pass my own test coverage
2. Intent to send several underlying architectural PRs that address the fundamental reasons this is happening
3. Ideally, if those changes were made this would pass more simply than it does now.

[drewcrawford]

After applying #4852 the regression should vanish.

[guybedford]

Please re-request a review when this is ready for another review.

[drewcrawford]

Test test_headless_worker_output_not_garbled for https://github.com/wasm-bindgen/wasm-bindgen/pull/4845#issuecomment-3629823055 now passes. Underlying problem was fixed in #4852.

cc: @guybedford @Spxg

[Spxg]

#[cfg(test)]
mod tests {
    use wasm_bindgen_test::console_log;
    wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_dedicated_worker);

    #[wasm_bindgen_test::wasm_bindgen_test]
    fn test() {
        console_log!("hello");
    }
}

Running headless tests in Chrome on `http://127.0.0.1:44889/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
Loading Wasm module...                            
using deprecated parameters for the initialization function; pass a single object instead
using deprecated parameters for the initialization function; pass a single object instead
running 1 test
hello
hello
test tests::test ... ok

There were duplicate outputs, seem introduced by #4852

[Spxg]

#[cfg(test)]
mod tests {
    use wasm_bindgen_test::console_log;
    wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_dedicated_worker);

    #[wasm_bindgen_test::wasm_bindgen_test]
    fn test() {
        console_log!("hello");
    }
}

NO_HEADLESS=1 cargo test --target wasm32-unknown-unknown

Run with headless miss "hello" output.

[drewcrawford]

If I understand correctly, we don't expect the log in case X, as it's considered too noisy

when	--nocapture	_
test passes	hello	X
test fails	hello	hello

Added four new tests to verify headless conforms to this matrix

[Spxg]

If I understand correctly, we don't expect the log in case X, as it's considered too noisy

Run with headless miss "hello" output.

Sorry, I said it wrong....what I meant to say is "Run with no headless and run_in_*_worker mode missing "hello" output in chrome DevTools.".

[drewcrawford]

Thanks, that issue should be fixed by special-casing non-headless mode here

[Spxg]

#[cfg(test)]
mod tests {
    use wasm_bindgen_test::console_log;
    wasm_bindgen_test::wasm_bindgen_test_configure!(run_in_dedicated_worker);

    #[wasm_bindgen_test::wasm_bindgen_test]
    fn test() {
        console_log!("hello");
    }
}

Running headless tests in Chrome on `http://127.0.0.1:37409/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
running 1 test                                    
test test ... ok

test result: ok. 1 passed; 0 failed; 0 ignored; 0 filtered out; finished in 0.00s
   Doc-tests playground                           

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

=>

Running headless tests in Chrome on `http://127.0.0.1:43651/`
Try find `webdriver.json` for configure browser's capabilities:
Not found
Loading Wasm module...                            
running 1 test
test test ... ok

   Doc-tests playgrounded; 0 failed; 0 ignored; 0 filtered out; finished in 0.00s                                                  

running 0 tests

test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.00s

There is text overlap. To be honest, you need more testing for run_in_*, and compare it with the release version.