uWebSockets icon indicating copy to clipboard operation
uWebSockets copied to clipboard

Automated browser testing, BrowserStack kind of deal. Smoke tests

Open ghost opened this issue 5 years ago • 6 comments

There should be more integration tests, checking things like different HTTP states and timeout.

ghost avatar May 17 '20 01:05 ghost

I just remembered we already have Hammer.js test for testing all kinds of different states. That test should be extended with pub/sub and more http. But more tests are needed either way.

ghost avatar May 30 '20 05:05 ghost

That's not it. We already have good security and stress tests (the server will not crash), and protocol standards tests (Autobahn pass). What is lacking is basic usage tests against browsers, simple tests that check things like idleTimeout, maxPayloadLength, close codes, sending/receiving messages. Basically, very simple tests that check things for basic functionality, not edge cases.

ghost avatar Jun 27 '20 01:06 ghost

Yep, I need this. Basic testing aginst browsers. For instance, I need to test basic messaging to/from Safari using the new support for x-webkit-deflate-frame and I have no such tests ready (and I don't have Safari).

Because tests are against browsers, you could make them entirely automatic so that browsers only need to navigate to the test, and then have the test server serve a page with the WS test in. Results could be printed on screen in the browser (and logged server side).

Every test would have its own URL, but share the same HTML page.

This way you could even automate this process with a service like BrowserStack so that a lot of browsers automatically go to this test address and run all the tests.

It's kind of like Autobahn, but more focused on checking browsers rather than checking standards (because some browsers simply do not follow standards).

ghost avatar Jan 13 '21 00:01 ghost

This can also test things like streaming huge files over Http, and the like. It doesn't have to only test WS features.

ghost avatar Jan 25 '21 14:01 ghost

https://www.browserstack.com/open-source?ref=pricing

ghost avatar May 19 '21 17:05 ghost

Something like the Crc32 example - very basic test that what you post has the correct hash, and similar. I need to add this. Note to self

uNetworkingAB avatar Oct 08 '22 06:10 uNetworkingAB

I began working on something incredibly simple that got a red background on failure and green on pass, it did a bunch of fetch requests here and there and things were looking pretty nice until I realized web browsers are a mine field; they implement parts of standards and have different limits on body size, and all kinds of strange incompatibilities that makes this idea of using the web browser as the test client completely unfeasible. I wanted to use BrowserStack to run through the test for all browsers, getting green on all, but yeah this is really not going to work.

Not even Chrome 105 has support to send chunked requests for http/1.1 - they have some limited support for doing so over http/2 but that's not what I am testing, Also Chrome has some really small limit on body size while Firefox does not. So the test doesn't even work on Chrome and Firefox doesn't even understand ReadableStream at all, it just sends the string "[object ReadableStream]" lol wtf I thought web programming would be simple but it's miserable.

So a better idea is to move this test over to Node.js and just have GitHub Actions run it automatically, then we at least get some form of smoke test that can reliably chug through all the different sizes and streams and whatnot, reliably.

uNetworkingAB avatar Nov 01 '22 02:11 uNetworkingAB

It was easy to port it to deno, nodejs and bun actually, so using all three of those for testing should be good enough

uNetworkingAB avatar Nov 01 '22 03:11 uNetworkingAB

Wow! There is now a smoke test that runs in GH actions by Deno and it works beautifully. This test will be run by Bun whenever that is possible, and same for Node.js. That's 3 implementations to test against, perfect

uNetworkingAB avatar Nov 01 '22 04:11 uNetworkingAB