jxl-oxide icon indicating copy to clipboard operation
jxl-oxide copied to clipboard

WebAssembly demo

Open nitroxis opened this issue 2 years ago • 4 comments

Hi,

I've created a small wasm demo based on jxl-oxide: https://gitlab.com/nitroxis/jxl-wasm. The main code is based on jxl-oxide-cli and works by transparently transcoding JXL files to PNG inside a ServiceWorker. From a user perspective, nothing other than including the jxl-worker.js script is required. One can simply use <img src="image.jxl" /> in HTML, similar to a polyfill.

Currently it requires the browser to put the entire JXL blob into the transcode function at once since there is no async or streaming API in jxl-oxide (or is there?), but it works alright nonetheless. Might not be a good idea for very big images due to memory constraints. It's also single-threaded currently. Successfully tested it in Firefox and MS Edge.

To try it out, one needs to have wasm-pack installed, then run wasm-pack build --target no-modules. Afterwards, run e.g. python -m http.server in the project root and open http://localhost:8000/demo/index.html, or use any other means of hosting the files via HTTP(S). ServiceWorkers do not work for file:// URLs, so HTTP is required, and HTTPS for anything but localhost.

I thought maybe you'd be interested to hear that this crate works well in wasm. Feel free to include this as an example or something.

nitroxis avatar Jul 09 '23 02:07 nitroxis

Great work, and thanks for sharing! I tested it briefly, and it worked quite well on reasonably sized images. A few thoughts:

  • I found it impressive that the binary size is less than 1 MiB. It's definitely not small for the web, but x86_64 build takes about 9 MiB.
  • It's ~2.5x slower on decoding bike conformance image compared to jxl-oxide-cli. Optimizing memory usage might improve decode speed, not sure.
  • Animations don't work somehow. I see jxl-wasm handles animation frames and APNG, but Firefox reports it as corrupted.

tirr-c avatar Jul 15 '23 15:07 tirr-c

Yes, binary size is indeed pretty small. I think wasm-pack applies some wasm optimizer on the compiler output, maybe that helps. As for performance, using SIMD in wasm would probably help in some areas. As far as I know, SIMD instructions aren't used currently. I've actually not tested animations, but I believe jxl-oxide-cli produces identical APNGs? I haven't really touched that part of the code.

A couple of things that would improve this demo/use case further:

  • It would be great to have async support, or at least some sort of incremental decoding API. WebAssembly (and JS in general) isn't really allowed to block the browser thread, so if jxl-oxide would support reading from e.g. AsyncRead, that would certainly help. It would also decrease memory usage since the entire JXL blob doesn't need to be buffered in RAM (on the JS side) first. It might also make things more responsive since the WASM worker can decode while the fetch request is still ongoing (i.e. pipelining it a bit).
  • JXL supports losslessly converting classic JPG images into JXL. As far as I understand, this process is reversible if no other JXL-specific features are used. It would be neat to support this here, since then it could just output a JPG instead of a PNG with no expensive re-encoding. Not sure how feasible this is, though.
  • Multithreading in wasm/web workers is possible in theory. Not sure how this could be used here (other than decoding multiple independent images simultaneously).
  • Color space handling is possibly incorrect since I've removed the code that used lcms2 because I didn't get it to compile to wasm. Not sure if a pure Rust alternative exists. Compiling Rust to wasm is a lot easier. ICC profiles should still be embedded in the PNG though.

nitroxis avatar Jul 15 '23 16:07 nitroxis

I was able to get the binary down to 821.7 KB using opt-level = "z" before compression and using wasm-opt to another 812.4 KB.

from here compressing with gzip -9 compresses it down to 326.3 KB and 294.7 KB using brotli -q 6 for the wasm

Quackdoc avatar Jul 16 '23 06:07 Quackdoc

I've actually not tested animations, but I believe jxl-oxide-cli produces identical APNGs? I haven't really touched that part of the code.

Ah, I remember png produces broken APNG when input is streamed... That's why jxl-oxide-cli buffers a whole frame. I'll create an issue when I have some free time.

tirr-c avatar Jul 18 '23 05:07 tirr-c