WebAssembly build

[FluorescentHallucinogen]

What about compiling FFMPEG to WebAssembly?

There are experimental native WebAssembly implementations for Google Chrome, Mozilla Firefox, Microsoft Edge and Apple Safari.

[Kagami]

I'll look into this, thanks. Though I don't think this will give big speedup compared to asm.js since it's just another format for the same thing. The really promising feature would be SIMD.js.

[FluorescentHallucinogen]

I mean compile FFMPEG directly to WebAssembly.

It is possible to compile any (LLVM-based) programming language other than JavaScript (initially mainly C/C++) to WebAssembly. See http://2ality.com/2015/06/web-assembly.html for more info.

[Kagami]

There is no real difference between compiling to asm.js or compiling to wasm. Most important advantage is load time, in terms of speed Firefox uses AOT compilation for asm.js anyway, so it shouldn't be faster.

I don't mean wasm is bad, just trying to say that pros are mostly related to initial loading/better browser support/standartization/etc.

[amilajack]

Judging by the wasm demo, it seems like there is a big performance difference.

[jfizz]

This might be a useful read: https://hacks.mozilla.org/2017/03/why-webassembly-is-faster-than-asm-js/. It highlights the performance differences between asm.js and WebAssembly.

[jfizz]

I am attempting to to give this a try (compiling to WebAssembly). First I built "ffmpeg-worker-webm.js" without any modifications to make sure I could successfully build on my machine. I was able to build and run that in my browser without any issues. To convert to WebAssembly, I took a look at https://github.com/kripken/emscripten/wiki/WebAssembly. I concluded that I needed to add the flag: -s WASM=1. Trying that in the browser leads to the following output in the console:

ffmpeg version n3.1.2 Copyright (c) 2000-2016 the FFmpeg developers
  test:66:13
  built with emcc (Emscripten gcc/clang-like replacement) 1.37.10 (commit 4ee1557a8f2473405971d4872bf71199174a2776)
  test:66:13
  configuration: --cc=emcc --enable-cross-compile --target-os=none --arch=x86 --disable-runtime-cpudetect --disable-asm --disable-fast-unaligned --disable-pthreads --disable-w32threads --disable-os2threads --disable-debug --disable-stripping --disable-all --enable-ffmpeg --enable-avcodec --enable-avformat --enable-avutil --enable-swresample --enable-swscale --enable-avfilter --disable-network --disable-d3d11va --disable-dxva2 --disable-vaapi --disable-vda --disable-vdpau --enable-decoder=vp8 --enable-decoder=vp9 --enable-decoder=theora --enable-decoder=mpeg2video --enable-decoder=mpeg4 --enable-decoder=h264 --enable-decoder=hevc --enable-decoder=png --enable-decoder=mjpeg --enable-decoder=vorbis --enable-decoder=opus --enable-decoder=mp3 --enable-decoder=ac3 --enable-decoder=aac --enable-decoder=ass --enable-decoder=ssa --enable-decoder=srt --enable-decoder=webvtt --enable-demuxer=matroska --enable-demuxer=ogg --enable-demuxer=avi --enable-demuxer=mov --enable-demuxer=flv --enable-demuxer=mpegps --enable-demuxer=image2 --enable-demuxer=mp3 --enable-demuxer=concat --enable-protocol=file --enable-filter=aresample --enable-filter=scale --enable-filter=crop --enable-filter=overlay --disable-bzlib --disable-iconv --disable-libxcb --disable-lzma --disable-sdl --disable-securetransport --disable-xlib --disable-zlib --enable-encoder=libvpx_vp8 --enable-encoder=libopus --enable-encoder=mjpeg --enable-muxer=webm --enable-muxer=ogg --enable-muxer=null --enable-muxer=image2 --enable-filter=subtitles --enable-libass --enable-libopus --enable-libvpx --extra-cflags=-I../libvpx/dist/include --extra-ldflags=-L../libvpx/dist/lib
  test:66:13
exception thrown: RuntimeError: integer overflow,wasm-function[5914]@http://localhost:9001/ffmpeg-worker-webm.js:7049272:1
wasm-function[5297]@http://localhost:9001/ffmpeg-worker-webm.js:7012313:1
wasm-function[1336]@http://localhost:9001/ffmpeg-worker-webm.js:5172982:1
wasm-function[4434]@http://localhost:9001/ffmpeg-worker-webm.js:6896029:1
wasm-function[2360]@http://localhost:9001/ffmpeg-worker-webm.js:6081367:1
ab/d._main@http://localhost:9001/ffmpeg-worker-webm.js:166:255
ab/d.qf@http://localhost:9001/ffmpeg-worker-webm.js:179:497
b@http://localhost:9001/ffmpeg-worker-webm.js:18:170
Sa@http://localhost:9001/ffmpeg-worker-webm.js:19:74
b@http://localhost:9001/ffmpeg-worker-webm.js:179:234
la@http://localhost:9001/ffmpeg-worker-webm.js:12:144
d@http://localhost:9001/ffmpeg-worker-webm.js:41:413
n/<@http://localhost:9001/ffmpeg-worker-webm.js:42:420
  ffmpeg-worker-webm.js:26:351
failed to asynchronously prepare wasm: RuntimeError: integer overflow  ffmpeg-worker-webm.js:26:351

@Kagami Do you happen to have any advice? I will admit, I am not terribly familiar with WebAssembly/asm.js, so I might be completely off the mark. I am just interested in the performance benefits that WebAssembly (potentially) offers.

[Elite]

@jfizz Here you go https://github.com/jonasof/videoconverter.js-wasm-demo/tree/master/wasm :)

[PaulKinlan]

That is interesting... I was suprised that WASM is considerably slower...

[mityok]

I'm getting more than twice the processing duration:

• about 6s video to snapshots on asm

• about 12s video to snapshots on wasm

very strange...

[shortercode]

I've been running tests on a fork of audioconverter.js and wasm doesn't seem to be giving much of an improvement, I haven't been able to create a proper benchmark for it yet but I've put some console.time calls in preRun and postRun. These are the results:

 [
  "File: Blink WASM: 13338.362426757812 ASM.js: 16333.235107421875 Diff: 82%",
  "File: Bad Seed WASM: 15875.986450195312 ASM.js: 18935.072998046875 Diff: 84%",
  "File: Dare WASM: 16816.656127929688 ASM.js: 19954.324462890625 Diff: 84%",
  "File: Cut Glass WASM: 17789.992553710938 ASM.js: 20735.35205078125 Diff: 86%",
  "File: sfx5 WASM: 110.224609375 ASM.js: 88.295166015625 Diff: 125%",
  "File: zelda_hey WASM: 31.7159423828125 ASM.js: 28.7606201171875 Diff: 110%",
  "File: zelda_item WASM: 218.906005859375 ASM.js: 171.7296142578125 Diff: 127%",
  "File: zelda_laugh WASM: 114.4654541015625 ASM.js: 85.4129638671875 Diff: 134%",
  "File: oglseby7 WASM: 11469.507934570312 ASM.js: 8620.97802734375 Diff: 133%",
  "File: zelda_menu WASM: 177.5545654296875 ASM.js: 87.14990234375 Diff: 204%",
  "File: zelda_shock WASM: 29.5728759765625 ASM.js: 20.3790283203125 Diff: 145%",
  "File: Phantom Lover WASM: 17242.292846679688 ASM.js: 15581.1845703125 Diff: 111%",
  "File: Simple Pleasures WASM: 15349.696533203125 ASM.js: 14238.008544921875 Diff: 108%"
]

Tests were converting wav files to webm opus ( audio only ), and each one was run twice.

Obviously needs more iterations and using something a little more accurate than console.time but it demonstrates the point. WASM proves to be fairly disappointing thus far in terms of performance improvements. Also chrome seems to crash occasionally with the WASM version, which is less than ideal.

[picitujeromanov]

Anybody still interested? Managed to compile it with O2 optimization. Compiling to wasm reduced size to 7mb, I have slight performance increase in some scenarios on Chrome and really nice performance boost on firefox ... my only remaining issue is that even with ALLOW_MEMORY_GROWTH=1 i still manage to crash chrome tab if I render 9 minute video with ultrafast preset and some filters

this build has added concat filter and mjpeg and image2 support for mp4_build as well ffmpeg-wasm3.zip

[XIAZY]

@picitujeromanov thank you for your build, it works! (and faster than asm.js with -O3 on my machine) do you have any advice on compiling it to wasm? thanks a lot

[picitujeromanov]

You can compile to wasm with my fork.

[XIAZY]

@picitujeromanov thank you. I really appreciate that.

[seranus]

@picitujeromanov using your mp4 worker and it works great, is there's a way to get ffmpg text command output?

[hperrin]

@picitujeromanov I'm building off your repo right now. I'm hoping this works. I have a project that actually requires transcoding video in Mobile Safari, and this is my last hope for getting this working.

[hperrin]

@picitujeromanov If you can believe it, this actually worked for me for a legitimate use case. I've got an end to end encrypted messenger app called Tunnelgram that supports sending videos. Normally a messenger app will take whatever video you give it and transcode it into something more compatible on the server side. Since mine is sending encrypted video, I can't transcode it on the server. So basically videos from Android were viewable by everyone, but videos from iPhone were only viewable on Apple devices. Your ffmpeg.js build saved me. Plus it's nice to not have to pay for the server cost of transcoding a bunch of videos. xD

Thank you @picitujeromanov and super thank you @Kagami!

[picitujeromanov]

happy to hear that

[picitujeromanov]

@hperrin have you tried encoding really big videos? I've had a problem where when I was using complex filters (picture in picture effect etc) it crashed the worker if the resulting video got to around 100mb in size. I see you updated ffmpeg and h264 lib, I might as well try my use case with your build now if it successfully finishes

[lieff]

I have no issues using WASM build too. May be it's filters code issue? I remember that working with filters is tricky with buffers retain. You can try execute your test locally, without emcc and check with -fsanitize=address (on linux also shows memory leaks, use drmemory if on windows). Leaks is very common with writing ffmpeg filters code.

[hperrin]

@picitujeromanov I have a limit of 20mb for videos, so I use a two pass strategy to transcode them with the right bitrates to be just under 20mb. The biggest video I've tried transcoding was 35mb, and it took over an hour. (I have all the fancy H.264 features enabled to make the quality really good. Basically equivalent to the "fast" preset, which wasn't working.)

These are the command line options: https://github.com/hperrin/tunnelgram/blob/master/app/src/Services/VideoService.js#L168

[hperrin]

I also took out every encoder but H.264 and AAC, since those are the only ones I'm using, and I wanted the file size to be as small as possible.

[Mathieu-Gosbee-ConnectedLab]

@hperrin have you tried compiling with -s ALLOW_MEMORY_GROWTH=1? That drastically reduced my transcoding time.

[hperrin]

@Mathieu-Gosbee-ConnectedLab Yeah, I've done that. I think the transcoding time is just because I'm using pretty high quality H.264 settings, and I'm using two passes (so it's almost twice as long just from that).

@picitujeromanov Scratch what I said earlier, I've tried it on a 53mb video and it worked no problem. I'll try it with a 160mb copy of Big Buck Bunny and upload the results. That is, if it doesn't crash, considering asking H.264 to bring a 160mb video down to 20mb is probably unrealistic.

[hperrin]

Ok, done. The original video was actually only 106mb, but here's the result: https://drive.google.com/open?id=19R9ZAWat6iF9HLqJ-77q39W1IXEYFj0K

And it took a little over an hour to do that one too.

[Laubeee]

@hperrin I would be much interested in your findings using WASM compared to asm.js and whether performance improved further with the newest libx264 as well Did you do any such benchmarks?

[tpetry]

A lot has happened in the last 4 years. So maybe @Kagami is interested on a fresh evaluation whether to add wasm implementation:

• asm.js is dead, it's still supported as it is an stricter subset than js but nobody optimize more for it (chrome even removed some asm.js optimizations)
• every browser is currently optimizing for webassembly because it's much easier than asm.js
• chrome does support the wasm threading standard so multi threaded transcodings would be possible if activated
• chrome is currently testing the wasm simd spec for webassembly, which could improve the performance by a lot too

So i modified the build scripts to build a specialized ffmpeg version (encoding to h264, no audio, some filters added) for me and received the following build sizes:

	Normal	Gzip --fast	Gzip --best
JS	9.07MB	2.18MB	1.73MB
Wasm	5.63MB	1.45MB	1.23MB

So when the files are gzipped there's not so much difference but if there is no gzipping the difference is huge. Nevertheless the whole build needs to be parsed by the interpreter. Parsing 9MB of JavaScript will take some time. Wasm has the added benefit that parsing is much more faster as the whole code is already transformed to a binary format and it's a lot smaller too.

But codesize is not everything, execution time is also very important. As my wasm build had a small bug i could not get rid off (returning the results failed everytime) i used the js and wasm builds of ffmpegjs/ffmpeg.js. As both are simply wrapping ffmpeg they should be pretty comparable. I built an easy reproducible benchmark which everybody can run. It may take a few minutes and i would suggest keeping the tab always in the foreground to not skew the results.

Here are my results on how much wasm was faster compared to a non-wasm version:

	Firefox	Chrome	Safari	Edge
macOS	6.51%	13.90%	FAIL*	-
Windows	8.65%	20.36%	-	-

As you can see the wasm version was always faster, sometimes by a large amount (chrome heavily optimizes for it, but i am not using a thread-enabled build!). A lot has happened on browser engines optimizing their wasm implementations 😀. It failed on Safari on parsing my ffmpeg paramters, don't know why, maybe a bug in the compiled version. I was not able to test on edge, but as it is nowadays simply chrome the numbers should be equal.

So in my opinion a lot has changed! Wasm shows very impressive performance benefits which should be used. I mean it's just another build. When a working wasm build is available it will be really interesting how performance will change for a wasm with threads build for chrome.

[Kagami]

Thanks for your research @tpetry. I agree, it's definitely worth to add support for WebAssembly.

[picitujeromanov]

This lady will give you more insight on how to build it for webAssembly, she even mentions you @Kagami and me https://www.youtube.com/watch?v=ziXYqUZqaEk

[tpetry]

Interestingly for wasm builds ALLOW_MEMORY_GROWTH could be re-enabled as there is no performance hit for this setting in wasm compared to asm.js. Many people are having maximum memory errors and have to fine tune the TOTAL_MEMORY so we maybe could get rid of this problem.