LibWeb+LibJS+LibWasm: Features desired/required for the Ruffle Flash Player emulator

[Lubrsi]

Features desired/required by the Ruffle Flash Player emulator:

• [x] Custom elements (for creating a player in DOM land, required)
• [ ] WebGL(2) (in place of CRC2D, desired)
• [ ] ReadableStream (for showing a loading bar when loading the Wasm file, required)
• [ ] WebAssembly SIMD extension (for loading and utilising an optimised Wasm blob, desired)
• [ ] Unknown LibWasm bug(s) causing a trap here when creating the player (needs further looking into, only required if the Wasm blob was optimized with Binaryen, otherwise desired)
• [ ] await spins forever on certain promises (needs re-looking into, I think it's mostly to do with the fetches) (required)
• [ ] AudioContext and certain AudioContext nodes (desired for playing audio, haven't looked into what specific nodes are wanted)
• [ ] URL.createObjectURL, URL.revokeObjectURL (for the user downloading the SWF file by the right click context menu, desired)
• [x] contextmenu event (for opening a custom context menu, desired)
• [ ] CanvasPattern#setTransform (for showing textures with CRC2D, required)
• [ ] TextDecoder constructor with ignoreBOM and fatal options set to true (whilst TextDecoder is required for pulling strings from Wasm memory, it seems these options are currently only desired)
• [ ] TextDecoder#decode accepting undefined input (required, as it creates a TextDecoder and immediately calls decode on it with no parameters outside of an unwind context)
• [ ] Wasm memory growth doesn't detach any ArrayBuffers returned by WebAssembly.Memory.buffer. (required, as it keeps a Uint8Array cache of the buffer, and it only knows to update the cache when TypedArray#byteLength becomes 0, which it does when the underlying ArrayBuffer is detached)
• [ ] pointerdown, pointermove and pointerup events (uses these in place of the mouse events, required)
• [ ] ImageData constructor accepting a Uint8ClampedArray (required for displaying textures with CRC2D)
• [ ] CRC2D.filter accepts SVG filter element references via url(#element_id) (desired)
• [ ] CRC2D.globalAlpha (desired)
• [ ] CRC2D.globalCompositeOperation (desired)
• [x] CRC2D.imageSmoothingEnabled (desired)
• [ ] CanvasFillRule with CRC2D.fill (desired)
• [ ] CRC2D.lineCap (desired)
• [ ] CRC2D.lineJoin (desired)
• [ ] CRC2D.miterLimit (desired)
• [ ] HTMLScriptElement.src must return the src attribute parsed relative to it's node document URL, then return that parsed relative URL as an absolute URL (required/desired, depends if the library files are in the same directory as ruffle.js. It gets this from document.currentScript.src). This is actually true of all attributes that are reflected as a URL, so this needs a more general audit.
• [ ] WebAssembly.instantiate(ArrayBuffer, object) fails overload resolution (required):

module.arrayBuffer().then(bytes => {
    // NOTE: module is a fetch() Response object
    WebAssembly.instantiate(bytes, imports).then(result => resolve(result)).catch(error => reject(error));
});

Unable to determine category for type named 'BufferSource', assuming it's an interface type.
Unable to determine category for type named 'Module', assuming it's an interface type.
Failed to determine IDL overload. (Probably because of unimplemented steps.)
Unhandled JavaScript exception (in promise): [TypeError] Overload resolution failed
  <unknown> at http://127.0.0.1:8081/core.ruffle.bbfb71e16568daaf348d.js:540:44
  <unknown> at http://127.0.0.1:8081/core.ruffle.bbfb71e16568daaf348d.js:540:52
  <unknown> at :0:0
  <unknown> at :0:0

This list is not exhaustive, I only tested Ruffle with Robot Unicorn Attack and Flight of the Hamsters.

There are many more issues with how we render flash files with Ruffle, but they have not been looked into to find what the specific issues are, for example:

https://user-images.githubusercontent.com/25595356/227027812-91fbd834-b5a8-4cf5-9fe3-398580b578d7.mp4

• The player is pushed far down the page and is very small
• Shows weird yellow shapes below the player
• Several rendering issues with CRC2D that haven't been looked into
• Unplayable performance

To see how the game is supposed to look, here's a game website that uses Ruffle: https://www.crazygames.com/game/robot-unicorn-attack

[alimpfard]

WebAssembly.Memory.buffer must return the same ArrayBuffer every time and must grow in place as Wasm's memory buffer is expanded (required, as Ruffle keeps a cached reference to this ArrayBuffer in a global variable and fully expects it to grow in place, as after a while memory indices will suddenly point outside of the buffer if not grown in place)

WebAssembly.Memory.buffer only returns a new wrapper ArrayBuffer object, the underlying buffer always points to the actual underlying bytebuffer and grows accordingly. Also, I don't think that attribute is supposed to be cacheable, see https://webassembly.github.io/spec/js-api/#dom-memory-grow:

1. Let store be the surrounding agent's associated store.
2. Let memaddr be this.[[Memory]].
3. Let ret be the mem_size(store, memaddr).
4. Let store be mem_grow(store, memaddr, delta).
5. If store is error, throw a RangeError exception.
6. Set the surrounding agent's associated store to store.
7. Reset the memory buffer of memaddr.
8. Return ret.

The "reset the memory buffer" step is supposed to replace the value of Memory.buffer.

[Lubrsi]

Ohh, you just reminded me that the actual issue is that typed arrays don't respond to it's underlying ArrayBuffer getting resized (it still has old size values, for example). Changed the comment.

[Lubrsi]

The actual actual issue is: Wasm memory growth doesn't detach any ArrayBuffers returned by WebAssembly.Memory.buffer. (required, as it keeps a Uint8Array cache of the buffer, and it only knows to update the cache when TypedArray#byteLength becomes 0, which it does when the underlying ArrayBuffer is detached)

[alimpfard]

Implemented that behaviour in #18079.

[torokati44]

Hi, Ruffle contributor here!

Awesome to see this being worked on, and even more amazing that it already runs "somewhat"! And thank you for the comprehensive list of stuff to do!

One minor correction: There are actually 5 WASM extensions needed/used by the "fancier" build, not just SIMD. See: https://github.com/ruffle-rs/ruffle/blob/b01e797e99e37b42ebe552d7958e2ba51d3125b7/web/packages/core/src/load-ruffle.ts#L45-L49

[alimpfard]

            bulkMemory(),
            simd(),
            saturatedFloatToInt(),
            signExtensions(),
            referenceTypes(),

Of the listed extensions, the only currently unimplemented one in LibWasm is SIMD;

• bulk memory was implemented in #17218
• non-trapping float-int conversions and sign extensions were implemented fairly early on in #7950
• reference types were part of the initial implementation in #7097.

I have an old WIP branch starting on SIMD (https://github.com/alimpfard/serenity/tree/wasm-simd), but that's probably drowning in conflicts by now :sweat_smile:

[torokati44]

Of the listed extensions, the only currently unimplemented one in LibWasm is SIMD;

Oh, wow, that's incredible! Wouldn't have thought that this much is done already (sorry), but should have checked!

[msub2]

After poking around with Audion on some Ruffle games it seems like the minimum needed from WebAudio would be:

• AudioNode
• AudioBuffer
• AudioBufferSourceNode
• AudioDestinationNode

Along with BaseAudioContext/AudioContext, which I am currently working on.