Any plan for EVAL_CTORS supporting in multi-threaded app?

[junyuecao]

trafficstars

There is an option for emcc to speed up the initializaton but it seems not supported for multi-threaded app. https://emscripten.org/docs/optimizing/Optimizing-Code.html#eval-ctors

Is threre any plan to support in multi-threaded app?

[kripken]

The error is

error: library_pthread.js: library_pthread.js:23: #error "EVAL_CTORS is not compatible with pthreads yet (passive segments)"

which is correct, I think. The ctor-evaller would need to support passive segments, that is, segments that it can't see their absolute location at compile time, somehow.

I am not aware of current work on this, but it would be good to add eventually, and there has been other work on the ctor-evaller recently (e.g. support for WasmGC).

[sbc100]

I think if we fix https://github.com/emscripten-core/emscripten/issues/12682 then I think this would be possible.

[msqr1]

Hmm @sbc100 , now that #12682 is fixed, can we maybe revisit this?

[sbc100]

I think this would require some binaryen changes, which is not really my wheel house. Perhaps @kripken can give an estimate on the amount of work it might be?

[kripken]

I think fixing that issue helped a lot here: Now the main module has no passive segments. Eval-ctors can now see the absolute position of all segments, and so it can run on them.

Trying this now, I see

$ ./emcc test/hello_world.c -sMAIN_MODULE -sEVAL_CTORS
building:INFO: ctor_evaller: trying to eval global ctors (--ctors=__wasm_call_ctors,main --kept-exports=main)
building:INFO: 

  ...stopping since could not create module instance: importGlobals: GOT.func.emscripten_glGenQueriesEXT

So to fix this we'd need to handle those GOT imports somehow. @sbc100 iirc those are references to functions that are not provided by the main module?

[sbc100]

So to fix this we'd need to handle those GOT imports somehow. [@sbc100](https://github.com/sbc100) iirc those are references to functions that are not provided by the main module?

Those are references to the global symbol table, which contains all global symbols, but ones defined in the main module and not.

The dynamic linker (in JS) is in charge of building that symbol table, and the GOT.XXX imports represent the addresses for function (in the table) and data (in the memory).

[kripken]

Then we'd need to do something like that table management, to handle MAIN_MODULE.

Though, that is a separate issue from pthreads, I think, which is the main topic here? So maybe #12682 doesn't help directly.

For pthreads, disabling the assert shows

building:INFO: ctor_evaller: trying to eval global ctors (--ctors=__wasm_call_ctors,main --kept-exports=main)
building:INFO: 

  ...stopping since could not flatten memory

The issue is that the Memory is defined in JS, and imported. Eval-ctors can't see the contents of an imported memory, so it can't "flatten" it into a form it can understand and modify. To support pthreads + eval-ctors, we'd need the memory to not be imported.

[sbc100]

Sadly imported-memory is essential to how threads work today (with instance-per-thread).

You could potentially setup an external memory (just like you would need to setup and external symbol table for dynamic linking), you could then potentially convert any changes to that external memory back into passive segments (either updating the existing ones or creating new ones). Sounds hard.