Running single threaded FFMPEG in a web worker

[hoomanaskari]

I know we all cried for a long time to have ffmpeg.wasm without the shared array buffer. It's now possible by forking the core and using github actions to generate the single threaded artifact.

The problem is this new core does not run in any workers at all, which is CPU blocking and freezes the UI (and sometimes the browser crashes).

I am not an expert with WASM and I am not sure if we can use web workers without having to use the sharedArrayBuffers at all?

Right now when try to use createFfmepg function in a worker (using Comlink), it says document is not defined, a clear sign and ffmpeg.js is not written to work in any thread other than the main one.

If this is possible at all (in the JS files), I would love to work on it too, given the right directions.

Thanks in advance.

[absurdprofit]

That error from my investigation is as a result of the usage of a package called resolve-url. Which as you can see is deprecated. Seeing as that is the case I don't see any other reason that ffmpeg cannot be used in a worker thread. resolve-url is supposed to be the browser version of path.resovle from node. I'm sure this behaviour can be achieved using modern APIs i.e. URL

[absurdprofit]

I forked ffmpeg.wasm and did a little refactoring to replace references of resolve-url with the new URL web API. You can view it here. FFmpeg now works in a web worker without any issues. So far all tests are passing and I'm doing a few tests on my own project to see if any of the resolved URLs fail. So far so good.

[tansanDOTeth]

@nxtexe does your fork fix #339?

[absurdprofit]

@nxtexe does your fork fix #339?

I'm not very sure. Do you have a minimal Example repo?

[absurdprofit]

@nxtexe does your fork fix #339?

I'm not very sure. Do you have a minimal Example repo?

Could it be related to? If it is that would suggest it might be an FFMPEG specific error and not just this port.

[Chris1234567899]

@nxtexe I tried to use your fork, but ran into problems with Chrome loading ffmpeg-core js file ( Not allowed to load local resource error) Can you confirm this is actually working?

[absurdprofit]

@nxtexe I tried to use your fork, but ran into problems with Chrome loading ffmpeg-core js file ( Not allowed to load local resource error)

Can you confirm this is actually working?

Yea I can. Can I see the error?

[Chris1234567899]

Not much to see: zone.js:1573 Not allowed to load local resource: file:///C:/projects/catr/src/assets/ffmpeg/ffmpeg-core.js

Looks like the error is thrown by ngzone (I'm in an angular project). Also, I copied the ffmpeg-core.js, ffmpeg-core.worker-js and ffmpeg-core.wasm to my assets as I made some edits bc of the proxy_main issue and single threading. So I use them as a file, not from a remote url.

But never mind actually, I'm currently rewriting the @ffmpeg/ffmpeg to typescript classes anyway, as it seems more convenient for my use case. Figured it out for main thread usage already, but not yet for angular workers... ;-)

[ImmortalZ]

That error from my investigation is as a result of the usage of a package called resolve-url. Which as you can see is deprecated. Seeing as that is the case I don't see any other reason that ffmpeg cannot be used in a worker thread. resolve-url is supposed to be the browser version of path.resovle from node. I'm sure this behaviour can be achieved using modern APIs i.e. URL

@nxtexe I have build with your fork branch，and get a file (name ffmpeg.min.js ). then I copy ffmpeg-core.js, ffmpeg-core.worker-js and ffmpeg-core.wasm to my assets .

Can you provide a demo that can run?

This mistake confused me. I got error like this:

My code:

[absurdprofit]

Where are you using postMessage?

[ImmortalZ]

Where are you using postMessage?

@nxtexe not my code throw error.

throw error code line:

[ImmortalZ]

@nxtexe It would be great if your branch could provide a demo and to show how to use

[absurdprofit]

@nxtexe It would be great if your branch could provide a demo and to show how to use

You were right it was acting a bit strange in the regular browsing environments. I'm not sure why. I tested using react and everything worked fine. But I changed how the package is compiled and added two new scripts for targeting webworkers. Anyway the example can be found here with instructions here. Hope this helps :D.

[Chris1234567899]

If anybody is using angular or typescript, I made a small port of the @ffmpeg/ffmpeg lib. https://gist.github.com/Chris1234567899/a00afe5e2de1beb1cb4053cbbafc4fe8

This works with web workers and single threaded ffmpeg.wasm now. So far no problems.

[ImmortalZ]

@nxtexe It would be great if your branch could provide a demo and to show how to use

You were right it was acting a bit strange in the regular browsing environments. I'm not sure why. I tested using react and everything worked fine. But I changed how the package is compiled and added two new scripts for targeting webworkers. Anyway the example can be found here with instructions here. Hope this helps :D.

@nxtexe

thk，great job! I found my code throw error because use 'Class' in work (i’m looking for the reason)

[riethmue]

@Chris1234567899 thank you so much for the port! I really struggled with making ffmpeg wasm working in a webworker in angular/ typescript.

[jimbojw]

What's the status of this? Can we get a version of ffmpeg wasm that works single-threaded in a Worker? I'm also experiencing the document is not defined problem.

[rossanmol]

Any updates on this please? :)

[ButzYung]

I was surprised that patch by @nxtexe was still not merged after months. Instead of waiting for the master branch or rebuilding the whole thing... I just directly edit the ffmepg.min.js, replace document with the right stuff for worker, and guess what, it works now and the whole process just took a few minutes lol

[ongnxco]

Hi @Chris1234567899 , I am using your script with Vite but when it comes to loading the ffmpeg-core file, I encounter this error.

Log {type: 'info', message: 'load ffmpeg-core'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'loading ffmpeg-core'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'fetch http://localhost:5173/ffmpeg/ffmpeg-core.js'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'http://localhost:5173/ffmpeg/ffmpeg-core.js file size = 106139 bytes'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'http://localhost:5173/ffmpeg/ffmpeg-core.js blob U…calhost:5173/df1c02fd-d3cc-4733-b6c7-b1362ea9ee96'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'fetch http://localhost:5173/ffmpeg/ffmpeg-core.wasm'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'http://localhost:5173/ffmpeg/ffmpeg-core.wasm file size = 24383038 bytes'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'http://localhost:5173/ffmpeg/ffmpeg-core.wasm blob…calhost:5173/444f9aa8-cda1-4aa7-b224-7bc7d60b37d6'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'fetch http://localhost:5173/ffmpeg/ffmpeg-core.worker.js'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'http://localhost:5173/ffmpeg/ffmpeg-core.worker.js file size = 3609 bytes'}
worker.js?type=module&worker_file:12 Log {type: 'info', message: 'http://localhost:5173/ffmpeg/ffmpeg-core.worker.js…calhost:5173/02fd0279-0816-4c22-9481-b9e835ede311'}
ffmpeg.js?t=1690046401730:225 Uncaught (in promise) TypeError: Failed to execute 'importScripts' on 'WorkerGlobalScope': Module scripts don't support importScripts().
    at ffmpeg.js?t=1690046401730:225:22
    at new Promise (<anonymous>)
    at FFmpeg.getCreateFFmpegCore (ffmpeg.js?t=1690046401730:224:16)
    at async FFmpeg.load (ffmpeg.js?t=1690046401730:21:19)
    at async onmessage (worker.js?type=module&worker_file:22:7)

[jumpjack]

Can anybody help with single-thread usage of FFMPEG?

As I am only converting single small images to test V360 filter, I don't need multithread and I don't even need high speed, but I can't get this simple page working:

https://github.com/jumpjack/browser-video-converter-ffmpeg.wasm/blob/main/mytest.html

I forked it from a page which processes videos, but I changed index.js so as to act on just a single image:

ffmpeg_params.push('-vf', 'v360=fisheye:e:ih_fov=200:iv_fov=200');    

I think I am importing the single-thread FFMPEG; but I keep ketting the sharedarray error.

[vovan1710]

You might find it helpful if you try to use it in Angular:

https://www.linkedin.com/pulse/web-based-video-conversion-from-webm-mp4-ffmpegwasm-hladchenko-wfejf/?trackingId=f2M1jZalQgG2QsAWLS4sYQ%3D%3D