New Emscripten audioWorklet draft

[Jonathhhan]

This draft should work as it is (without any additional changes). It works well, but maybe its not implemented nicely. Besides that: The main downside is, that sharedArrayBuffer needs to be enabled with -S USE_PTHREADS and because of that CORS header need to be set, if uploaded to a server: https://github.com/emscripten-core/emscripten/issues/20454 The sharedArrayBuffer and the other audioWorklet related Emscripten flags are also only needed, if ofSoundStream is used. But what speaks for audioWorklets is, that it performs much better and the old ScriptProcessorNode method https://developer.mozilla.org/en-US/docs/Web/API/ScriptProcessorNode is deprecated and will be removed at some point.

Here is the main part of the implementation: https://github.com/Jonathhhan/openFrameworks/blob/audioWorklet-3/addons/ofxEmscripten/src/ofxEmscriptenSoundStream.cpp

Maybe this PR is good for testing and thinking about a proper implementation?

[ofTheo]

Could we use this approach for CORS and bundle that js file in the folder package? https://stackoverflow.com/a/72628095

ping @Jonathhhan - any thoughts?

[ofTheo]

Hey @Jonathhhan

could you give this a try and see if it solves the CORS issue? https://stackoverflow.com/a/72628095

[Jonathhhan]

@ofTheo Yes, I will give it a try on Thursday. Maybe earlier. Actually there is a threading problem with ofxPd and USE_PTHREADS=1 or Audioworklets, that I tried to solve (not really successful), which has to to with Pure Data's sys_lock: https://github.com/pure-data/pure-data/issues/2104

Generally, sys_lock and sys_unlock is not safe to use from within an external as it can always deadlock.

But without Pure Data (libpd) Audioworklets work really well (at least I did not had any issues).

[Jonathhhan]

@ofTheo

could you give this a try and see if it solves the CORS issue?

Yes, it does work. Only flag I still need to set in the .htaccess file is: AddType application/wasm wasm (should not be a problem to add that line to the script). Her an update with my audioWorklet issue: https://github.com/emscripten-core/emscripten/issues/20617

[ofTheo]

@Jonathhhan - that's awesome! if you want to update this PR with those changes - I'll merge it in.

[Jonathhhan]

@ofTheo maybe I first make a pr that solves the CORS issue? The audioWorklet issue is described here: https://github.com/emscripten-core/emscripten/issues/20617 And it makes it impossible to use ofxPd without hacks (which is the fault of Pure Data's sys_lock system). Perhaps it makes sense to wait with this PR. As soon as the audioWorklet mutex is implemented it will work. Another option would be to have the possibility to switch between audioWorklet and ScriptProcessorNode.

[Jonathhhan]

@ofTheo it seems that the recent Pure Data / ofxPd commits solved my issue and ofxPd is now working well with audioWorklets. My only thought: Since it is necessary to enable pthreads, it is more complicated (mainly the CORS thing), so maybe it would be nice to have the scriptProcessorNode as the standard and audioworklets as an option (in the best case automatically enabled, if -s AUDIO_WORKLET=1 is set)? scriptProcessorNode on the other hand will be deprecated at some point, so maybe we need to switch sooner or later...

[Jonathhhan]

Here is an updated branch: https://github.com/Jonathhhan/openFrameworks/tree/emscripten_3.1.57_audioWorklet Maybe someone can have a look, if there is a more elegant way to implement audioWorklets. But I tested it and it works well.