evmone icon indicating copy to clipboard operation
evmone copied to clipboard

cmake: Add wasi toolchain

Open axic opened this issue 3 years ago • 11 comments

Part of #293.

axic avatar May 20 '21 22:05 axic

With cmake -DTOOLCHAIN=wasi .. this goes pretty far, but it seems that imported ethash project currently depends on pthreads, which is not supported:

Scanning dependencies of target ethash
[ 62%] Building CXX object lib/ethash/CMakeFiles/ethash.dir/managed.cpp.o
[ 62%] Building CXX object lib/ethash/CMakeFiles/ethash.dir/ethash.cpp.o
[ 87%] Building CXX object lib/ethash/CMakeFiles/ethash.dir/progpow.cpp.o
[ 87%] Building C object lib/ethash/CMakeFiles/ethash.dir/primes.c.o
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:26:6: error: no type named 'mutex' in namespace 'std'
std::mutex shared_context_mutex;
~~~~~^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:30:6: error: no type named 'mutex' in namespace 'std'
std::mutex shared_context_full_mutex;
~~~~~^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:47:10: error: no member named 'lock_guard' in namespace 'std'
    std::lock_guard<std::mutex> lock{shared_context_mutex};
    ~~~~~^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:47:26: error: no member named 'mutex' in namespace 'std'
    std::lock_guard<std::mutex> lock{shared_context_mutex};
                    ~~~~~^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:47:33: error: use of undeclared identifier 'lock'; did you mean 'clock'?
    std::lock_guard<std::mutex> lock{shared_context_mutex};
                                ^~~~
                                clock
/Users/alex/Projects/evmone/wasi-sdk-12.0//share/wasi-sysroot/include/time.h:61:9: note: 'clock' declared here
clock_t clock (void);
        ^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:68:10: error: no member named 'lock_guard' in namespace 'std'
    std::lock_guard<std::mutex> lock{shared_context_full_mutex};
    ~~~~~^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:68:26: error: no member named 'mutex' in namespace 'std'
    std::lock_guard<std::mutex> lock{shared_context_full_mutex};
                    ~~~~~^
/Users/alex/.hunter/_Base/135567a/60efa9e/32bc5aa/Build/ethash/Source/lib/ethash/managed.cpp:68:33: error: use of undeclared identifier 'lock'; did you mean 'clock'?
    std::lock_guard<std::mutex> lock{shared_context_full_mutex};
                                ^~~~
                                clock
/Users/alex/Projects/evmone/wasi-sdk-12.0//share/wasi-sysroot/include/time.h:61:9: note: 'clock' declared here
clock_t clock (void);
        ^
8 errors generated.
make[5]: *** [lib/ethash/CMakeFiles/ethash.dir/managed.cpp.o] Error 1
make[5]: *** Waiting for unfinished jobs....
make[4]: *** [lib/ethash/CMakeFiles/ethash.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [ethash-Release-prefix/src/ethash-Release-stamp/ethash-Release-build] Error 2
make[1]: *** [CMakeFiles/ethash-Release.dir/all] Error 2
make: *** [all] Error 2

Can ethash be improved to not compile pthread-dependent parts for evmone?

axic avatar May 20 '21 22:05 axic

The second problem is building evmc, dl* used by the loader is not available. This would mean we need some configurability what is built out of evmc as a dependency for evmone (excluding tests/tools).

axic avatar May 20 '21 22:05 axic

Codecov Report

Merging #328 (1724d3d) into master (2c1f473) will not change coverage. The diff coverage is n/a.

:exclamation: Current head 1724d3d differs from pull request most recent head a320c66. Consider uploading reports for the commit a320c66 to get more accurate results

@@           Coverage Diff           @@
##           master     #328   +/-   ##
=======================================
  Coverage   99.77%   99.77%           
=======================================
  Files          27       27           
  Lines        3932     3932           
=======================================
  Hits         3923     3923           
  Misses          9        9           
Flag Coverage Δ
consensus 97.73% <ø> (ø)
unittests 99.77% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

codecov[bot] avatar May 20 '21 22:05 codecov[bot]

D'oh, there's a toolchain+platform file in wasi-sdk, but that seems to be working worse for me.

axic avatar May 20 '21 22:05 axic

Can ethash be improved to not compile pthread-dependent parts for evmone?

This is the problem Silkworm is also having (cc @yperbasis, @AndreaLanfranchi). We can attack the ethash library first, probably by disabling some "managed" API in wasm build.

Here we only need the ethash::keccak library so other solution is to not build full ethash library. This can be done by adding some CMake options to ethash or not using Hunter to get it.

chfast avatar May 21 '21 07:05 chfast

Yes we experienced the same problem in Silkworm and opted to amalgamate in our source (with slight modifications) only the strict necessary from ethash library code. Here what we have now https://github.com/torquem-ch/silkworm/tree/master/ethash/ethash

AndreaLanfranchi avatar May 21 '21 08:05 AndreaLanfranchi

This is the problem Silkworm is also having (cc @yperbasis, @AndreaLanfranchi). We can attack the ethash library first, probably by disabling some "managed" API in wasm build.

Here we only need the ethash::keccak library so other solution is to not build full ethash library. This can be done by adding some CMake options to ethash or not using Hunter to get it.

I think for the evmone case, we may be better off having a BUILD_ETHASH=OFF setting (which is on by default), as we only need keccak.

Not sure about the Silkworm case, @AndreaLanfranchi does silkworm also uses the ethash part for PoW?

axic avatar May 22 '21 10:05 axic

But for evmc we may need to have cmake settings for building the client- vs vm-side only. On the vm-side the loader is not needed. Another dumb option is just having an ifdef in the loader for unsupported platforms resulting in abort(), but I guess that is not to your liking.

axic avatar May 22 '21 10:05 axic

@axic

Not sure about the Silkworm case, @AndreaLanfranchi does silkworm also uses the ethash part for PoW?

Yes we implement the full Ethash PoW validation

AndreaLanfranchi avatar May 22 '21 10:05 AndreaLanfranchi

Currently blocked on evmc:

[  5%] Building C object evmc/lib/instructions/CMakeFiles/instructions.dir/instruction_metrics.c.o
[ 10%] Building C object evmc/lib/instructions/CMakeFiles/instructions.dir/instruction_names.c.o
[ 21%] Building CXX object evmc/lib/tooling/CMakeFiles/tooling.dir/run.cpp.o
[ 21%] Building C object evmc/lib/loader/CMakeFiles/loader.dir/loader.c.o
/home/builder/project/evmc/lib/loader/loader.c:25:10: fatal error: 'dlfcn.h' file not found
#include <dlfcn.h>
         ^~~~~~~~~
1 error generated.

Why do we need to even build these from evmc? At this point I think mostly only the headers are used.

axic avatar Feb 06 '23 17:02 axic

Why do we need to even build these from evmc? At this point I think mostly only the headers are used.

These are used for evmc testing tool but we can skip it.

chfast avatar Feb 06 '23 18:02 chfast