openFrameworks Emscripten 3.1.19

This is a bit(e) size pull request for Emscripten 3.1.19. Here are some additional details: https://forum.openframeworks.cc/t/emscripten-3-1-19/40294 Most of the changes in ofxAppEmscriptenWindow.cpp are from @roymacdonald I left out all the Audioworklet changes, but as soon as they are part of the main Emscripten branch, it would be nice to implement them: https://github.com/emscripten-core/emscripten/pull/16449

Aug 21 '22 23:08 Jonathhhan

I guess, the checks are not successful because I added -s DYNCALLS=1 which is needed for audio input and output (heap access?).

Aug 25 '22 05:08 Jonathhhan

@Jonathhhan does it mean we need to change the emscripten version used by the CI here?

https://github.com/openframeworks/openFrameworks/blob/master/.github/workflows/build-emscripten.yml#L24

The rest of the PR looks good to me.

Aug 25 '22 17:08 ofTheo

@ofTheo not sure, because I build the libs directly, without docker.

Aug 25 '22 23:08 Jonathhhan

@Jonathhhan I guess I am wondering, what the DYNCALLS=1 means in terms of OF. Do we need to build all libraries OF uses for emscripten with a certain flag, is it just one library?

Or is it just about passing the flag to emscripten ( right now the flag is not recognized ). Maybe its just the emscripten version that needs updating?

Could you push to your branch with a change to this line: https://github.com/openframeworks/openFrameworks/blob/master/.github/workflows/build-emscripten.yml#L24

from:

$PWD:/src emscripten/emsdk:1.40.0 bash

to

$PWD:/src emscripten/emsdk:3.1.19 bash

Then the OF CI will be running the same version as you 🙂

Aug 26 '22 17:08 ofTheo

@ofTheo I made the change in https://github.com/openframeworks/openFrameworks/blob/master/.github/workflows/build-emscripten.yml#L24 I actually compiled the libs with that flag, but it does not seem to make a difference (the mentioned changes are still necessary). And maybe I did not use that file at all. I compiled with something like: -t emscripten update all core libs DYNCALL is only needed for PLATFORM_LDFLAGS = -Wl --gc-sections --preload-file bin/data@data --emrun --bind --profiling-funcs -s USE_FREETYPE=1 -s DYNCALLS=1, the libs do not need that flag. And I can only confirm that -s DYNCALLS=1 is needed for the audio input and output of ofxPd, but I guess it could be necessary for other audio processing, too. This is the part in library_html5audio.js where it seems to be needed:

		var inbufferArray = Module.HEAPF32.subarray(inbuffer>>2,(inbuffer>>2)+bufferSize*inputChannels);
		var outbufferArray = Module.HEAPF32.subarray(outbuffer>>2,(outbuffer>>2)+bufferSize*outputChannels);

Aug 26 '22 22:08 Jonathhhan

Thanks @Jonathhhan

I think maybe I'll need to build the apothecary stuff against 3.1.19 and get those updated in the apothecary repo and then hopefully this PR will compile fine :)

Aug 26 '22 23:08 ofTheo

Not sure if it changes anything but I've noticed 3.1.20 is out

Aug 27 '22 16:08 dimitre

@dimitre yes, it makes sense to compile with the latest version.

Aug 27 '22 21:08 Jonathhhan

I replaced dynCall('viiii',callback, [bufferSize,inputChannels,outputChannels,userData]); with {{{ makeDynCall('viiii', 'callback') }}}(bufferSize,inputChannels,outputChannels,userData); in library_html5audio.js. With that DYNCALLS=1 is not needed anymore.

Aug 27 '22 23:08 Jonathhhan

I made a first attempt to implement the new audio worklet branch (see the end of the Emscripten link in the first post of this thread). Not working yet, but should be doable. Maybe someone more experienced can have a look. It is not part of this pull request.

Aug 28 '22 17:08 Jonathhhan

I made a first working example with the current audioWorklet branch. It is very hacky and I needed to edit some of the Emscripten files. It may be also possible to implement it much better and without editing the emscripten files. But maybe it is helpful as a first step. https://github.com/Jonathhhan/openFrameworks/tree/emscripten_3.1.9_audioWorklet/addons/ofxEmscripten It works with this Emscripten branch: https://github.com/juj/emscripten/tree/audio_worklets audio_worklet.js and library_webaudio.js need to be copied into emscripten/src and webaudio.h and html5audio.h need to be copied into emscripten/system/include/emscripten And 3.1.21 is released...

Sep 08 '22 14:09 Jonathhhan

Re-running the emscripten job now that Freeimage is updated in apothecary.

@Jonathhhan @themancalledjakob - if this passes let me know if you think its good to merge or if there is any small fixes cleanup that needs to happen.

Nov 17 '22 18:11 ofTheo

Currently erroring with a linking error:

File "/emsdk/upstream/emscripten/tools/building.py", line 578, in parse_llvm_nm_symbols
    status = line[entry_pos + 11] # Skip address, which is always fixed-length 8 chars.
IndexError: string index out of range

This seems to suggest that maybe apothecary might need to be building libs with the newer SDK? https://github.com/emscripten-core/emscripten/issues/17826

Nov 17 '22 18:11 ofTheo

Currently erroring with a linking error:
File "/emsdk/upstream/emscripten/tools/building.py", line 578, in parse_llvm_nm_symbols
    status = line[entry_pos + 11] # Skip address, which is always fixed-length 8 chars.
IndexError: string index out of range
This seems to suggest that maybe apothecary might need to be building libs with the newer SDK? emscripten-core/emscripten#17826

@ofTheo thats probably the case, maybe because they updated llvm in 3.1.25: https://github.com/emscripten-core/emscripten/blob/main/ChangeLog.md

Nov 18 '22 00:11 Jonathhhan

Okay now with libs in apothecary being built with 3.1.21 this looks like it all works. Only thing I am not sure is if we need to provide an option for returning EMSCRIPTEN_WEBGL_CONTEXT_HANDLE similar to how we used to do.

Maybe I'll merge this in for now and we can do some smaller PRs for other things.

@Jonathhhan does this fix the fullscreen issues?

Nov 18 '22 22:11 ofTheo

@ofTheo now it includes the fullscreen fix (also added this change: https://github.com/openframeworks/openFrameworks/issues/7152).

Nov 19 '22 01:11 Jonathhhan

Thanks @Jonathhhan !!

Nov 19 '22 01:11 ofTheo

@ofTheo maybe just update it for 3.1.26 (if it is not difficult - but I guess that only the apothecary libs need to be recompiled. I use 3.1.25 at the moment)?

Nov 19 '22 02:11 Jonathhhan

Out of curiosity I just tried compiling all apothecary libs and this branch with emsdk 1.40.0, and except using emrun to launch a dev server everything seems to just work. running something like php -S 0.0.0.0:8080 in bin was able to launch the project successfully though, so hopefully there is no real big issue with being backwards compatible.

of course otherwise it's better to just compile with 3.1.25 or 3.1.26 (it seems as if there are is a new version almost every other week atm)

Nov 19 '22 09:11 themancalledjakob

to get a better overview over what works and what doesn't, I thought it would be good to have a gallery of all examples.

I made a little sketch of how this may work, it's super rough. but it might help with getting an overview. I uploaded it here with the current status. compiled with emsdk 3.1.25 + downloaded libs from apothecary

if you want to try it yourself you need to download this: https://pointer.click/files/testAllExamples.zip and put it in your $OF_ROOT/scripts/emscripten directory. then run testAllExamples.sh it will create a gallery with all examples in there, a bit threejs style (but much simpler). you can click on the examples to load them in the preview iframe. if there is a preview image, it will use it in the list. otherwise it will use a grey placeholder image. but if there was an error compiling, the preview image will be red and the error output from make will be loaded in the iframe instead of the example.

note: some examples also have runtime errors when loading them. these errors are not red, you'd have to open the example to catch them. and of course, some examples cannot work, but some could possibly be fixed.

also I noticed there are some glitches and the whole thing could be more elegant, but what do you think, can this be useful?

Nov 19 '22 16:11 themancalledjakob

@themancalledjakob thanks. I think this is really useful for finding out what is working (I already recognized some examples that were working before, like the 3DPrimitivesExample).

Nov 19 '22 17:11 Jonathhhan

The 3DPrimitivesExample works if I comment out this line:

function _html5video_grabber_pixel_format(id) {
    // return allocate(intArrayFromString(VIDEO.grabbers[id].pixelFormat), "i8", ALLOC_STACK)
}

Otherwise this is the error message: ALLOC_STACK is a library symbol and not included by default; add it to your library.js __deps or to DEFAULT_LIBRARY_FUNCS_TO_INCLUDE on the command line (e.g. -sDEFAULT_LIBRARY_FUNCS_TO_INCLUDE=$ALLOC_STACK)

Uncaught TypeError: allocate is not a function
    at _html5video_grabber_pixel_format (3DPrimitivesExample.js:formatted:8809:12)
    at ofxEmscriptenVideoGrabber::getPixelFormat() const (3DPrimitivesExample.wasm:0x3220c)
    at ofVideoGrabber::setPixelFormat(ofPixelFormat) (3DPrimitivesExample.wasm:0x4ae8a)
    at ofVideoGrabber::setup(int, int, bool) (3DPrimitivesExample.wasm:0x4aa35)
    at ofApp::setup() (3DPrimitivesExample.wasm:0x22f50)
    at ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) (3DPrimitivesExample.wasm:0x4d5a5)
    at std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) (3DPrimitivesExample.wasm:0x94f58)
    at ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) (3DPrimitivesExample.wasm:0x5e4b7)
    at ofCoreEvents::notifySetup() (3DPrimitivesExample.wasm:0x5e375)
    at ofxAppEmscriptenWindow::loop() (3DPrimitivesExample.wasm:0x2f209)
_html5video_grabber_pixel_format @ 3DPrimitivesExample.js:formatted:8809
$ofxEmscriptenVideoGrabber::getPixelFormat() const @ 3DPrimitivesExample.wasm:0x3220c
$ofVideoGrabber::setPixelFormat(ofPixelFormat) @ 3DPrimitivesExample.wasm:0x4ae8a
$ofVideoGrabber::setup(int, int, bool) @ 3DPrimitivesExample.wasm:0x4aa35
$ofApp::setup() @ 3DPrimitivesExample.wasm:0x22f50
$ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) @ 3DPrimitivesExample.wasm:0x4d5a5
$std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) @ 3DPrimitivesExample.wasm:0x94f58
$ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) @ 3DPrimitivesExample.wasm:0x5e4b7
$ofCoreEvents::notifySetup() @ 3DPrimitivesExample.wasm:0x5e375
$ofxAppEmscriptenWindow::loop() @ 3DPrimitivesExample.wasm:0x2f209
$std::__2::__function::__func<void (*)(), std::__2::allocator<void (*)()>, void ()>::operator()() @ 3DPrimitivesExample.wasm:0x94a0e
$main @ 3DPrimitivesExample.wasm:0x21afe
(anonymous) @ 3DPrimitivesExample.js:formatted:975
callMain @ 3DPrimitivesExample.js:formatted:10163
doRun @ 3DPrimitivesExample.js:formatted:10208
(anonymous) @ 3DPrimitivesExample.js:formatted:10217
setTimeout (async)
run @ 3DPrimitivesExample.js:formatted:10213
runCaller @ 3DPrimitivesExample.js:formatted:10151
removeRunDependency @ 3DPrimitivesExample.js:formatted:941
processPackageData @ 3DPrimitivesExample.js:formatted:308
getRequest.onsuccess @ 3DPrimitivesExample.js:formatted:269

Nov 19 '22 18:11 Jonathhhan

wow - this is super @themancalledjakob and something that could be great to have hosted on the site ( once the kinks are worked out ).

makes seeing potential issues easier to see. would be awesome if we could somehow redirect the console error messages to the main console to make it easier to track down.

Nov 19 '22 20:11 ofTheo

cool :) i'm happy this is useful!

would be awesome if we could somehow redirect the console error messages to the main console to make it easier to track down.

which main console do you mean?

Nov 22 '22 11:11 themancalledjakob

The 3DPrimitivesExample works if I comment out this line:

function _html5video_grabber_pixel_format(id) {
    // return allocate(intArrayFromString(VIDEO.grabbers[id].pixelFormat), "i8", ALLOC_STACK)
}

Otherwise this is the error message: ALLOC_STACK is a library symbol and not included by default; add it to your library.js __deps or to DEFAULT_LIBRARY_FUNCS_TO_INCLUDE on the command line (e.g. -sDEFAULT_LIBRARY_FUNCS_TO_INCLUDE=$ALLOC_STACK)

Uncaught TypeError: allocate is not a function
    at _html5video_grabber_pixel_format (3DPrimitivesExample.js:formatted:8809:12)
    at ofxEmscriptenVideoGrabber::getPixelFormat() const (3DPrimitivesExample.wasm:0x3220c)
    at ofVideoGrabber::setPixelFormat(ofPixelFormat) (3DPrimitivesExample.wasm:0x4ae8a)
    at ofVideoGrabber::setup(int, int, bool) (3DPrimitivesExample.wasm:0x4aa35)
    at ofApp::setup() (3DPrimitivesExample.wasm:0x22f50)
    at ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) (3DPrimitivesExample.wasm:0x4d5a5)
    at std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) (3DPrimitivesExample.wasm:0x94f58)
    at ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) (3DPrimitivesExample.wasm:0x5e4b7)
    at ofCoreEvents::notifySetup() (3DPrimitivesExample.wasm:0x5e375)
    at ofxAppEmscriptenWindow::loop() (3DPrimitivesExample.wasm:0x2f209)
_html5video_grabber_pixel_format @ 3DPrimitivesExample.js:formatted:8809
$ofxEmscriptenVideoGrabber::getPixelFormat() const @ 3DPrimitivesExample.wasm:0x3220c
$ofVideoGrabber::setPixelFormat(ofPixelFormat) @ 3DPrimitivesExample.wasm:0x4ae8a
$ofVideoGrabber::setup(int, int, bool) @ 3DPrimitivesExample.wasm:0x4aa35
$ofApp::setup() @ 3DPrimitivesExample.wasm:0x22f50
$ofNode::onParentOrientationChanged(glm::qua<float, (glm::qualifier)0>&) @ 3DPrimitivesExample.wasm:0x4d5a5
$std::__2::__function::__func<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&), std::__2::allocator<std::__2::shared_ptr<of::priv::Function<ofKeyEventArgs, std::__2::recursive_mutex>> ofEvent<ofKeyEventArgs, std::__2::recursive_mutex>::make_function<ofMainLoop>(ofMainLoop*, void (ofMainLoop::*)(ofKeyEventArgs&), int)::'lambda'(void const*, ofKeyEventArgs&)>, bool (void const*, ofKeyEventArgs&)>::operator()(void const*&&, ofKeyEventArgs&) @ 3DPrimitivesExample.wasm:0x94f58
$ofEvent<ofEventArgs, std::__2::recursive_mutex>::notify(ofEventArgs&) @ 3DPrimitivesExample.wasm:0x5e4b7
$ofCoreEvents::notifySetup() @ 3DPrimitivesExample.wasm:0x5e375
$ofxAppEmscriptenWindow::loop() @ 3DPrimitivesExample.wasm:0x2f209
$std::__2::__function::__func<void (*)(), std::__2::allocator<void (*)()>, void ()>::operator()() @ 3DPrimitivesExample.wasm:0x94a0e
$main @ 3DPrimitivesExample.wasm:0x21afe
(anonymous) @ 3DPrimitivesExample.js:formatted:975
callMain @ 3DPrimitivesExample.js:formatted:10163
doRun @ 3DPrimitivesExample.js:formatted:10208
(anonymous) @ 3DPrimitivesExample.js:formatted:10217
setTimeout (async)
run @ 3DPrimitivesExample.js:formatted:10213
runCaller @ 3DPrimitivesExample.js:formatted:10151
removeRunDependency @ 3DPrimitivesExample.js:formatted:941
processPackageData @ 3DPrimitivesExample.js:formatted:308
getRequest.onsuccess @ 3DPrimitivesExample.js:formatted:269

oh, cool! good you found that, there is a similar error with quite some other examples

Nov 22 '22 11:11 themancalledjakob

which main console do you mean?

The sort of fake terminal that OF outputs messages to below the renderer. Not sure if that is possible, but making the js errors more visible could help track down the bugs ( maybe even with a toggle which is default off ).

Nov 22 '22 15:11 ofTheo

hmmm not having tested it, but it should be possible to override console.error as seen here: https://stackoverflow.com/questions/69460871/how-to-catch-console-error-so-title

console.errorOriginal = console.error;
console.error = (...messages) => {
    console.errorOriginal(...messages); 
    // add messages to oF terminal
}

could that work?

Nov 22 '22 16:11 themancalledjakob

openFrameworks openFrameworks copied to clipboard

Emscripten 3.1.19

openFrameworks
openFrameworks copied to clipboard