Dynamically creating a module in JS, function not found from linked module via dynamicLibraries

[cggallant]

I've created a repo with code that reproduces the issue I'm having: https://github.com/cggallant/emscripten-sample-code-for-ticket/tree/main/dynamicLibraries

I have a main module (core.wasm in this case) that is built with the Modularize=1 flag so that it can be created dynamically by the JS.

• The code for the module (core.cpp) has one function ValidateValueProvided.

The command line:

emcc core.cpp -s MAIN_MODULE=2 -s MODULARIZE=1 -s "EXPORTED_FUNCTIONS=['_strlen']" -s "EXPORTED_RUNTIME_METHODS=['ccall']" -o core.js

The other module (product.wasm in this case) is a side module

• The code for the module (product.cpp) has one function ValidateName that calls into the core module's ValidateValueProvided function.

The command line:

emcc product.cpp -s SIDE_MODULE=2 -O1 -o product.wasm

The JS (test.js) causes the module to be created and specifies the dynamicLibraries to load. Here it happens at page load but, in my actual code, it happens when the user interacts with certain elements on the page:

Module({ dynamicLibraries: ['product.wasm'] }).then((module) => {
  productModule = module;
});

Later, a ccall is made on that generated module to the ValidateName function of the side module.

Something changed starting with version 3.1.31 where the code no longer recognizes the side module's function when I use ccall. The browser's debugger shows that the side module was loaded and I see the function in it.

The browser's console gets the following error message when I try to call the function:

Uncaught RuntimeError: Aborted(Assertion failed: Cannot call unknown function ValidateName, make sure it is exported)
    abort http://localhost:5000/core.js:915
    assert http://localhost:5000/core.js:392
    getCFunc http://localhost:5000/core.js:4572
    ccall http://localhost:5000/core.js:4616
    validate http://localhost:5000/test.js:14
    onclick http://localhost:5000/test.html:1

If I build the modules using 3.1.30 or earlier, the call to the side module works.

I'm just wondering if perhaps I need a different flag when building the modules now or perhaps I need to pass something else into the Module object when triggering the creation in the JS?

Thanks

[sbc100]

Yes, the way symbols were handled in dyanmic linking was changed in #18540.

The idea is that the Module object should only contain the specifically EXPORTED_FUNCTIONS. All other symbols are internal the program and not exported on the Module object (although they were prior to #18540).

Looking at how ccall is implemented it looks like it only works for EXPORTED_FUNCTIONS: https://github.com/emscripten-core/emscripten/blob/667505546e57040aaf00a56c07c0b924158307e9/src/library_ccall.js#L10

What happens if you add ValidateName to EXPORTED_FUNCTIONS? Does that work? If that doesn't work we can look into fixing it, or maybe we could have ccall also look at the internal symbol table and not just the outer Module object. In any case we are clearly missing a test case here.

Also, BTW, if you pass product.wasm when linking the main module then you don't need to specify dynamicLibraries and you will also get all of your side module dependencies added to them main module automatically.

[cggallant]

Specifying the ValidateName function for the side module didn't work for me.

Could it be the order that things are placed? The main module having functions that the side module will be using.

Once linked, rather than the JS calling a function in the main module, the side module's function is what gets called.

The idea was that the main module was almost like a common library. As the program runs, a new Module object is created by the JS from the core.wasm and linked to that area's side module. The logic for that section is in the side module.

---

Actually, thinking about it... I'll try flipping their roles (changing the side modules into main modules and the main module into a side module). Rather than having one main module and several side modules, I'll have several main modules and one side module.

[cggallant]

I'll need to do a bit more testing but switching the roles around seems to have done the trick (the common logic is now the side module and the code the JS calls into is the main module).

I've created a branch of the repo with my changes for reference if you're interested: https://github.com/cggallant/emscripten-sample-code-for-ticket/tree/swapped_roles/dynamicLibraries

[sbc100]

Specifying the ValidateName function for the side module didn't work for me.

Sorry I'm not sure I understand what you mean by this. Can you elaborate a litte?

My recommendation was to add any functions that you need to ccall to EXPORTED_FUNCTIONS, did you try that?

[cggallant]

I tried that first. The code that was being compiled as a side module (product.cpp) already had the function decorated with EMSCRIPTEN_KEEPALIVE so adding the function to the EXPORTED_FUNCTIONS list at the command line didn't change anything.

As of 3.1.31, using dynamicLibraries to link two or more modules on the fly only seems to expose functions to ccall if the function is in the main module (it seems that functions in the side modules can now only be called by the main module's code and not by the JS code). Realizing that, I switched around the modules that were main and those that were side and I was able to get my code to work.

The idea with why I have things linked at run-time using dynamicLibraries was that a common library would be used and then sections of the app would be loaded to take advantage of that common library.

If it's possible to have the side module's exported functions accessible by the JS, that would be ideal but I have verified that adjusting my code works.

[cyk2018]

@sbc100 seems this problem still.

[cyk2018]

The link author has posted up can run, but in the Chapter8 https://github.com/cggallant/WebAssembly-in-Action/tree/master/updated-code/Chapter%208/8.1%20EmDynamicLibraries, this program can not run well. I don't know if the problem is this code define one common function in main_module and call it by cpp in two side_module. I use EXPORTED_FUNCTIONS but it don't work well.

[Splizard]

I believe this is the same issue as https://github.com/emscripten-core/emscripten/issues/21491