Emscripten doesn't compile

[jerbob92]

I'm trying to compile the project using the build-emscripten.sh script to compile to WebAssembly. My goal is to build it into a server-side WebAssembly so that it can be directly embedded in go-libheif (without using CGO).

However, the script always results in the following errors, I have tried multiple versions, including the version in the install script and the latest version, am I doing something wrong?

wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_class
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_class
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_class
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_bindings
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_function
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_enum
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_enum_value
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_enum
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_enum_value
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_enum
wasm-ld: error: ../libheif/libheif.a(heif.cc.o): undefined symbol: _embind_register_enum_value
wasm-ld: error: too many errors emitted, stopping now (use -error-limit=0 to see all errors)

[jerbob92]

It looks like this might have to do with the use of the (very) old fastcomp backend. Are there any plans to move to the LLVM backend?

[jerbob92]

Found out it needs the very specific version 1.37.26 of fastcomp. When compiling with that it does work. It has the same errors (unresolved symbol) but it just seems to ignore those.

[jerbob92]

This is what I ended up with to use the latest SDK and latest backend to build a standalone WASM:

CONFIGURE_ARGS="-DENABLE_MULTITHREADING_SUPPORT=OFF -DWITH_GDK_PIXBUF=OFF -DWITH_EXAMPLES=OFF -DBUILD_SHARED_LIBS=ON -DENABLE_PLUGIN_LOADING=OFF"
#export PKG_CONFIG_PATH="${DIR}/libde265-${LIBDE265_VERSION}"

emcmake cmake -DCMAKE_TOOLCHAIN_FILE=$EMSDK/upstream/emscripten/cmake/Modules/Platform/Emscripten.cmake $CONFIGURE_ARGS \
    -DCMAKE_EXE_LINKER_FLAGS="-lde265" \
    -DLIBDE265_INCLUDE_DIR="${DIR}/libde265-${LIBDE265_VERSION}" \
    -DLIBDE265_LIBRARY="-L${DIR}/libde265-${LIBDE265_VERSION}/libde265/.libs"
emmake make -j${CORES}

echo "Running Emscripten..."
LIBHEIFA="libheif/libheif.a"
EXPORTED_FUNCTIONS=$($EMSDK/upstream/bin/llvm-nm $LIBHEIFA --format=just-symbols | grep "^heif_\|^de265" | grep "[^:]$" | sed 's/^/_/' | paste -sd "," -)

emcc "$LIBHEIFA" \
    -s EXPORTED_FUNCTIONS="$EXPORTED_FUNCTIONS,_free,_malloc" \
    -s LLD_REPORT_UNDEFINED \
    -s WASM=1 \
    -s ALLOW_MEMORY_GROWTH=1 \
    -s STANDALONE_WASM=1 \
    -s EXPORTED_RUNTIME_METHODS='["ccall", "cwrap"]' \
    -s ERROR_ON_UNDEFINED_SYMBOLS=0 \
    -O3 \
    -std=c++11 \
    -L${DIR}/libde265-${LIBDE265_VERSION}/libde265/.libs \
    -lde265 \
    -o libheif.wasm \
    --no-entry

Not sure yet if it fully works, will keep you updated.

[muellerkyle]

I'm trying to get this to work too. How did you determine that version of fastcomp to be the compatible version? Was it based upon the date of the emscripten script here in libheif?

As an aside, I'm trying to get this to run within a docker container:

docker run \
  --rm \
  -v $(readlink -f ~/repos/github.com/strukturag/libheif/):/src \
  -u $(id -u):$(id -g) \
  emscripten/emsdk:1.39.0 \
  /src/build-emscripten.sh

Not having much like so far! Slowly working my way back through emsdk versions.

[jerbob92]

@muellerkyle They use version 3.1.29 of the SDK, which is defined in https://github.com/strukturag/libheif/blob/master/scripts/install-emscripten.sh And they use version 1.37.26 of fastcomp, which is defined in https://github.com/strukturag/libheif/blob/master/.github/workflows/emscripten.yml

However, I'm pretty sure you can get it to work with the default backend now, but it will probably need some extra changes compared to mine since they are specific to standalone WASM.

[jerbob92]

@muellerkyle I got the webversion to work with the latest emscripten too (with the default backend), but it does require loads of changes to the build script and javascript. I'll clean up the changes soon and make a PR, not sure if they want the javascript to change this much though.

[muellerkyle]

@jerbob92 Thank you! I've not seen the script to install emcripten as it wasn't referenced in the build script. This library is pretty new to me.

Maybe I'm on the wrong path using the docker image that is published by emscripten and I should use the libheif install script to build a custom emscripten image.

[farindk]

Thank you for looking into this. A PR would be very welcome.

[jerbob92]

@fancycode since most of the work on this has been done by you, can you help me out a bit? Can you tell me the reasoning behind the following options?

 --closure 0 \
-s NO_EXIT_RUNTIME=1 \
-s TOTAL_MEMORY=${TOTAL_MEMORY} \
-s ASSERTIONS=0 \
-s INVOKE_RUN=0 \
-s DOUBLE_MODE=0 \
-s PRECISE_F32=0 \
-s DISABLE_EXCEPTION_CATCHING=1 \
-s USE_CLOSURE_COMPILER=0 \
-s LEGACY_VM_SUPPORT=1 \
--memory-init-file 0 \

I kinda want to keep the build as default as possible and leave these all out.

Regarding the pre.js / post.js, how attached are you to the current interface of the libheif JS client? To get it to work I had to move a lot of stuff around because the embind stuff is now not loaded soon enough to make the current implementation work.

[jerbob92]

@farindk In my own copy I also include AOM, is that something you would be interested in adding to the script?

[farindk]

@jerbob92 Yes, sure, great. We should enable this with a compile option, though, and at the same time make h265 support optional.

[jerbob92]

@farindk @fancycode @muellerkyle MR is here: #925

[farindk]

Thanks, @jerbob92 I have added an additional option to compile to plain JS instead of WASM.

[farindk]

I reorganized how the data is passed around (794a791b5048ff2bcdb6507c4413719652f568cf). Instead of using the hack with passing data through strings, it is now returning raw arrays to the source data and then copying this on the client side. This copy has to be done anyways because the lifespan of the image object is limited and we have to remove the stride-padding in many cases.

[jerbob92]

I reorganized how the data is passed around (794a791). Instead of using the hack with passing data through strings, it is now returning raw arrays to the source data and then copying this on the client side. This copy has to be done anyways because the lifespan of the image object is limited and we have to remove the stride-padding in many cases.

Nice! That will probably make the decoding speed a lot faster. I do notice that it only uses the heif_channel_interleaved now, will this always work because the selected chroma heif_chroma_interleaved_RGBA.

[farindk]

I do notice that it only uses the heif_channel_interleaved now, will this always work because the selected chroma heif_chroma_interleaved_RGBA.

Yes, all the conversion is now done inside libheif. This is not necessarily faster, but it takes more care that the correct equations are used.

The current approach is still wasteful if we do not want RGB output, but YCbCr. For example, when converting to JPEG, it would be better to get YCbCr directly. If this is a relevant use-case, we should add another function to get these without conversion.

[jerbob92]

I do notice that it only uses the heif_channel_interleaved now, will this always work because the selected chroma heif_chroma_interleaved_RGBA.

Yes, all the conversion is now done inside libheif. This is not necessarily faster, but it takes more care that the correct equations are used.

The current approach is still wasteful if we do not want RGB output, but YCbCr. For example, when converting to JPEG, it would be better to get YCbCr directly. If this is a relevant use-case, we should add another function to get these without conversion.

Cool :) I do wonder what the current advantage is of the embind methods, since they basically just call the C methods and pass along the output, wouldn't it be easier to just do that from the JS methods?