CMake Toolkit: Shared Library as Side Module

[ax3l]

Enable support to build shared libraries as side modules. This should simplify builds with CMake that currently have to reside to manual link steps.

X-ref:

• https://github.com/emscripten-core/emscripten/issues/15276#issuecomment-1039349267
• #16157
• #8362 cc @sbc100 @kripken
• https://github.com/pyodide/pyodide/pull/2169

[kripken]

Thanks @ax3l !

I don't know much about CMake, but the flags seem right.

However there are test failures here. Let us know if you need help debugging those.

[ax3l]

Thanks a lot! Let's see if keeping the default in emscripten to "static" already solves this.

If you have pointers where we can add a new test for building a shared library, then I would add that to the PR :)

[ax3l]

I cannot see why the tests are failing, is there an easy way to increase verbosity in CI or run them locally?

[kripken]

Running them locally should be easy, see the docs which are here: https://emscripten.org/docs/getting_started/test-suite.html

For example, ./tests/runner other.test_cmake_js will run one of the failures.

For more details you can read the test code in tests/test_other.py (search for that test name) and see what it is doing.

[ax3l]

Related upstream issue: https://gitlab.kitware.com/cmake/cmake/-/issues/23237 I would like to allow building shared libs, but keep defaulting to static libs.

[ax3l]

I see when executing test_cmake_js that some symbols cannot be found. (This one builds a shared library now.)

['/home/axel/src/emscripten/emcmake', 'cmake', '-DCMAKE_BUILD_TYPE=Debug', '-DCMAKE_VERBOSE_MAKEFILE=ON', '-G', 'Unix Makefiles', '/home/axel/src/emscripten/tests/cmake/target_js']
em++: warning: ignoring unsupported linker flag: `-soname` [-Wlinkflags]
em++: warning: please replace -g4 with -gsource-map [-Wdeprecated]
em++: warning: ignoring unsupported linker flag: `-rpath` [-Wlinkflags]
error: undefined symbol: _Z20cpp_library_functionv (referenced by top-level compiled C/C++ code)
warning: Link with `-s LLD_REPORT_UNDEFINED` to get more information on undefined symbols
warning: To disable errors for undefined symbols use `-s ERROR_ON_UNDEFINED_SYMBOLS=0`
warning: __Z20cpp_library_functionv may need to be added to EXPORTED_FUNCTIONS if it arrives from a system library
Error: Aborting compilation due to previous errors

I am not sure why, is there another flag we should add to compile/link of shared libs? :) -sEXPORT_ALL?

I also tried -sEXPORTED_FUNCTIONS=_Z20cpp_library_functionv -sEXPORT_ALL without success on the cpp_lib/lib.cpp test target.

[ax3l]

Nice! If we can make the default of BUILD_SHARED_LIBS=OFF work I would support this change.

Thanks! Turns out, that works :)

It's only a few libs in the tests where we explicitly do shared builds that fail now. Before this, the property TARGET_SUPPORTS_SHARED_LIBS FALSE overwrote add_library(... SHARED) calls to be static builds.

We need to decide for the failing tests if we declare the exports or keep them to static.

[ax3l]

For test_dlfcn_self (test_core.core0) I am not sure why __data_end is not exported now :thinking:

[sbc100]

For test_dlfcn_self (test_core.core0) I am not sure why __data_end is not exported now thinking

A rebase should fix that issue I think.

[ax3l]

Thx, rebased :)

I see this in my PR for tests:

1 workflow awaiting approval
First-time contributors need a maintainer to approve running workflows. [Learn more.](https://docs.github.com/actions/managing-workflow-runs/approving-workflow-runs-from-public-forks) 12 expected, 2 pending, and 4 successful checks

Can someone please approve me? :)

[sbc100]

Whats the deal with CMAKE_STRIP FALSE... if that doesn't work can you open a bug report.. and leave a comment in the place where we disable this?

[ax3l]

The CMAKE_STRIP issue I saw in https://github.com/pyodide/pyodide/pull/2169 when building a pybind11 extension: https://github.com/pybind/pybind11/blob/v2.9.1/tools/pybind11Common.cmake#L399-L411

Simply did not work with what it picked for CMAKE_STRIP. Do you have your own strip command that needs to be used instead here?

[sbc100]

The CMAKE_STRIP issue I saw in pyodide/pyodide#2169 when building a pybind11 extension: https://github.com/pybind/pybind11/blob/v2.9.1/tools/pybind11Common.cmake#L399-L411

Simply did not work with what it picked for CMAKE_STRIP. Do you have your own strip command that needs to be used instead here?

Is the CMAKE_STRIP separate/unrelated to this one? If so, can leave it out of this change for now?

llvm-strip is supposed to work and I believe it does, but I'll need to take a look at the specific issue here.

[ax3l]

Oh, it definitely did not pick that one. Let's set it here. Let me try a path.

Update: uses now emstrip - thank you for #16431 :sparkling_heart:

[anutosh491]

Hi, I have been using these flags for my project separately but it would also be great if support to build shared libraries as side modules is supported from emscripten's side. Is some movement expected here to maybe finish of the pending work if any ?

[Andrej730]

Hi! Any updates on this?

Took me awhile to figure out how pyodide is hacking into emscripten to produce dynamic python extensions (patched toolchain file in pyodide).

Would be great if this can be integrated into emscripten itself - it would simplify the process and would allow existing cmake configs with shared libs to support cross-compilation into wasm just by adding emcmake.