Reduce code duplication in tests

[jeluard]

          The amount of copy pasted code is quite high in this file, maybe something that could be improved in this PR (or later one) to make it easier to add new events in the future

Originally posted by @nflaig in https://github.com/ChainSafe/lodestar/pull/6391#discussion_r1482020391

[FaybianB]

@jeluard @philknows Hey, may I be assigned this issue?

[jeluard]

Sure!

[FaybianB]

@jeluard So I'd like to validate that my changes were effective and the tests still pass but I notice that running the e2e tests from unstable (without my changes) fails.

Any advice? Here's the result from yarn test:e2e:

yarn run v1.22.21
$ lerna run test:e2e --concurrency 1
(node:1024) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
lerna notice cli v7.3.0

 >  Lerna (powered by Nx)   Running target test:e2e for 7 projects:

    - @lodestar/beacon-node
    - @chainsafe/lodestar
    - @lodestar/logger
    - @lodestar/params
    - @lodestar/prover
    - @lodestar/spec-test-util
    - @lodestar/validator

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

> @lodestar/params:"test:e2e"

@lodestar/params: $ vitest --run --config vitest.e2e.config.ts --dir test/e2e/
@lodestar/params:  RUN  v1.2.1 /Users/ugongetthisword/Miscellaneous/lodestar/packages/params
@lodestar/params:  ✓ test/e2e/ensure-config-is-synced.test.ts  (2 tests) 611ms
@lodestar/params:  ✓ test/e2e/overridePreset.test.ts  (2 tests) 396ms
@lodestar/params:  ✓ test/e2e/setPreset.test.ts  (2 tests) 368ms
@lodestar/params:  Test Files  3 passed (3)
@lodestar/params:       Tests  6 passed (6)
@lodestar/params:    Start at  21:25:11
@lodestar/params:    Duration  1.60s (transform 56ms, setup 23ms, collect 69ms, tests 1.38s, environment 0ms, prepare 33ms)

> @lodestar/logger:"test:e2e"

@lodestar/logger: $ vitest --run --config vitest.e2e.config.ts --dir test/e2e
@lodestar/logger:  RUN  v1.2.1 /Users/ugongetthisword/Miscellaneous/lodestar/packages/logger
@lodestar/logger: test-log-worker
@lodestar/logger: test-log-worker
@lodestar/logger:  ✓ test/e2e/logger/workerLogs.test.ts  (3 tests) 391ms
@lodestar/logger:  Test Files  1 passed (1)
@lodestar/logger:       Tests  3 passed (3)
@lodestar/logger:    Start at  21:25:14
@lodestar/logger:    Duration  766ms (transform 77ms, setup 17ms, collect 225ms, tests 391ms, environment 0ms, prepare 33ms)

> @lodestar/validator:"test:e2e"

@lodestar/validator: $ vitest --run --config vitest.e2e.config.ts --dir test/e2e
@lodestar/validator:  RUN  v1.2.1 /Users/ugongetthisword/Miscellaneous/lodestar/packages/validator
@lodestar/validator: (node:1088) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
@lodestar/validator: (Use `node --trace-deprecation ...` to show where the warning was created)
@lodestar/validator:  ❯ test/e2e/web3signer.test.ts  (14 tests | 1 skipped) 695ms
@lodestar/validator: ⎯⎯⎯⎯⎯⎯ Failed Suites 1 ⎯⎯⎯⎯⎯⎯⎯
@lodestar/validator:  FAIL  test/e2e/web3signer.test.ts > web3signer signature test
@lodestar/validator: Error: Could not find a working container runtime strategy
@lodestar/validator:  ❯ getContainerRuntimeClient ../../node_modules/testcontainers/src/container-runtime/clients/client.ts:61:9
@lodestar/validator:  ❯ GenericContainer.start ../../node_modules/testcontainers/src/generic-container/generic-container.ts:81:20
@lodestar/validator:  ❯ Module.startExternalSigner ../test-utils/src/externalSigner.ts:46:28
@lodestar/validator:  ❯ test/e2e/web3signer.test.ts:46:22
@lodestar/validator:      44| 
@lodestar/validator:      45|     const password = "password";
@lodestar/validator:      46|     externalSigner = await startExternalSigner({
@lodestar/validator:        |                      ^
@lodestar/validator:      47|       keystoreStrings: await getKeystoresStr(
@lodestar/validator:      48|         password,
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[1/2]⎯
@lodestar/validator:  FAIL  test/e2e/web3signer.test.ts > web3signer signature test
@lodestar/validator: TypeError: Cannot read properties of undefined (reading 'container')
@lodestar/validator:  ❯ test/e2e/web3signer.test.ts:61:26
@lodestar/validator:      59| 
@lodestar/validator:      60|   afterAll(async () => {
@lodestar/validator:      61|     await externalSigner.container.stop();
@lodestar/validator:        |                          ^
@lodestar/validator:      62|   });
@lodestar/validator:      63| 
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯[2/2]⎯
@lodestar/validator: ⎯⎯⎯⎯⎯⎯ Unhandled Errors ⎯⎯⎯⎯⎯⎯
@lodestar/validator: 
@lodestar/validator: Vitest caught 1 unhandled error during the test run.
@lodestar/validator: This might cause false positive tests. Resolve unhandled errors to make sure your tests are not affected.
@lodestar/validator: ⎯⎯⎯⎯ Unhandled Rejection ⎯⎯⎯⎯⎯
@lodestar/validator: Error: Cannot find module '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node'
@lodestar/validator: Require stack:
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/bindings.js
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/lib.js
@lodestar/validator:  ❯ Function.Module._resolveFilename node:internal/modules/cjs/loader:1149:15
@lodestar/validator:  ❯ Function.Module._load node:internal/modules/cjs/loader:990:27
@lodestar/validator:  ❯ Module.require node:internal/modules/cjs/loader:1237:19
@lodestar/validator:  ❯ require node:internal/modules/helpers:176:18
@lodestar/validator:  ❯ Object.<anonymous> ../../node_modules/@chainsafe/blst/src/bindings.ts:3:27
@lodestar/validator:  ❯ Module._compile node:internal/modules/cjs/loader:1378:14
@lodestar/validator:  ❯ Object.Module._extensions..js node:internal/modules/cjs/loader:1437:10
@lodestar/validator:  ❯ Module.load node:internal/modules/cjs/loader:1212:32
@lodestar/validator:  ❯ Function.Module._load node:internal/modules/cjs/loader:1028:12
@lodestar/validator:  ❯ Module.require node:internal/modules/cjs/loader:1237:19
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
@lodestar/validator: Serialized Error: { code: 'MODULE_NOT_FOUND', requireStack: [ '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/bindings.js', '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/lib.js' ] }
@lodestar/validator: This error originated in "test/e2e/web3signer.test.ts" test file. It doesn't mean the error was thrown inside the file itself, but while it was running.
@lodestar/validator: ⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
@lodestar/validator:  Test Files  1 failed (1)
@lodestar/validator:       Tests  1 skipped (14)
@lodestar/validator:      Errors  1 error
@lodestar/validator:    Start at  21:25:15
@lodestar/validator:    Duration  1.69s (transform 448ms, setup 15ms, collect 853ms, tests 695ms, environment 0ms, prepare 32ms)
@lodestar/validator: error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

 —————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————————

 >  Lerna (powered by Nx)   Running target test:e2e for 7 projects failed

   Tasks not run because their dependencies failed or --nx-bail=true:
   
   - @lodestar/beacon-node:test:e2e
   - @chainsafe/lodestar:test:e2e
   - @lodestar/prover:test:e2e
   - @lodestar/spec-test-util:test:e2e
   
   Failed tasks:
   
   - @lodestar/validator:test:e2e

error Command failed with exit code 1.
info Visit https://yarnpkg.com/en/docs/cli/run for documentation about this command.

I've tried yarn install to ensure I had all the dependencies but that also fails with a bunch of errors.

[nflaig]

@lodestar/validator: Error: Cannot find module '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node'
@lodestar/validator: Require stack:
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/bindings.js
@lodestar/validator: - /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/lib.js

Looks like blst is not installed. Did you install all prerequisites noted in MacOS Specifics?

[FaybianB]

Yes, I did all those prerequisites. However, I didn't do these:

• 🌐 Run a beacon node on mainnet or a public testnet.
• 💻 Utilize the whole stack by starting a local testnet.

Are these necessary to getting the e2e tests running?

./scripts/run_e2e_env.sh start also fails:

Wait for the node to be ready
Waiting for 0.0.0.0:5001.........................................................................................................................
Timeout

[nflaig]

Are these necessary to getting the e2e tests running?

No, but first you should make sure the installation (yarn install) doesn't fail and you can build the project (yarn build). From there you can do a sanity check via ./lodestar --help (see Getting Started). If that works you can attempt to run the tests.

./scripts/run_e2e_env.sh start also fails

I remember we recently increased the timeout of this script, and I think the first time it will pull images which might take a bit. Running it a second time should resolve the timeout issue.

I've tried yarn install to ensure I had all the dependencies but that also fails with a bunch of errors.

Can you please share the errors here or on discord (#lodestar-developer channel) which is a better place to figure out what's the issue on your end.

[FaybianB]

@nflaig Thanks, I'd like to ask in the Discord but I haven't found a valid invite (the one you linked is invalid as well).

[nflaig]

@nflaig Thanks, I'd like to ask in the Discord but I haven't found a valid invite (the one you linked is invalid as well).

The README invite and this one both work for me. I also newly created the one I linked in the previous comment.

[FaybianB]

@nflaig I'm not sure what the issue is but I get invalid for both.

Nonetheless, I'll post the errors here in the meantime:

yarn install v1.22.21
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Building fresh packages...
[11/18] ⢀ @chainsafe/blst
[15/18] ⢀ c-kzg
[12/18] ⢀ bcrypto
[14/18] ⢀ cpu-features
error /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst: Command failed.
Exit code: 1
Command: node dist/scripts/install.js
Arguments: 
Directory: /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst
Output:
Retrieving BLST native bindings /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node ...
(node:40510) [DEP0040] DeprecationWarning: The `punycode` module is deprecated. Please use a userland alternative instead.
(Use `node --trace-deprecation ...` to show where the warning was created)
Error: Error importing BLST native bindings: 404 Not Found
    at Object.downloadReleaseAsset (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/scripts/downloadReleaseAsset.js:22:15)
    at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
    at async Object.downloadBindings (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/scripts/downloadBindings.js:9:5)
    at async install (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/dist/scripts/install.js:34:9)
Building BLST native bindings /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/darwin-arm64-120-binding.node from source...
BLST_WRAP_CPP_PREBUILD /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/blst_wrap.cpp exists, SWIG will be skipped
Launching node-gyp {
  nodeJsExec: '/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node',
  nodeGypExec: '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js',
  cwd: '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js',
  BLST_WRAP_CPP_PREBUILD: '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/prebuild/blst_wrap.cpp'
}
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info find Python using Python version 3.11.7 found at "/opt/homebrew/opt/[email protected]/bin/python3.11"
gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.11
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  ACTION binding_gyp_blst_target_blst_wrap Release/obj.target/blst/geni/blst_wrap.cpp
/bin/sh: python: command not found
make: *** [Release/obj.target/blst/geni/blst_wrap.cpp] Error 127
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command "/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node" "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js
gyp ERR! node -v v21.6.1
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok 
Error: Error building BLST native bindings: Command failed: /Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js rebuild
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info find Python using Python version 3.11.7 found at "/opt/homebrew/opt/[email protected]/bin/python3.11"
gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.11
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
/bin/sh: python: command not found
make: *** [Release/obj.target/blst/geni/blst_wrap.cpp] Error 127
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/lib/build.js:194:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command "/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node" "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js" "rebuild"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/blst/bindings/node.js
gyp ERR! node -v v21.6.1
gyp ERR! node-gyp -v v8.4.1
gyp ERR! not ok 

    at genericNodeError (node:internal/errors:984:15)
    at wrappedFn (node:internal/errors:538:14)
    at ChildProcess.exithandler (node:child_process:422:12)
    at ChildProcess.emit (node:events:519:28)
    at maybeClose (node:internal/child_process:1105:16)
    at Socket.<anonymous> (node:internal/child_process:457:11)
    at Socket.emit (node:events:519:28)
    at Pipe.<anonymous> (node:net:337:12) {
  code: 1,
  killed: false,
  signal: null,
  cmd: '/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/@chainsafe/blst/node_modules/node-gyp/bin/node-gyp.js rebuild'




warning Error running install script for optional dependency: "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
Arguments: 
Directory: /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info find Python using Python version 3.11.7 found at \"/opt/homebrew/opt/[email protected]/bin/python3.11\"
gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.11
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_aarch64_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_arm_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_mips_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_ppc_linux.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_freebsd.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_macos.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_windows.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/filesystem.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/stack_line_reader.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/string_view.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/hwcaps.o
  LIBTOOL-STATIC Release/cpu_features.a
  CXX(target) Release/obj.target/cpufeatures/src/binding.o
In file included from ../src/binding.cc:1:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/node.h:73:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8.h:24:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-array-buffer.h:12:
/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-local-handle.h:253:5: error: static_assert failed due to requirement 'std::is_base_of<v8::Value, v8::Data>::value' \"type check\"
    static_assert(std::is_base_of<T, S>::value, \"type check\");
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../nan/nan_callbacks_12_inl.h:175:20: note: in instantiation of function template specialization 'v8::Local<v8::Value>::Local<v8::Data>' requested here
      cbinfo(info, obj->GetInternalField(kDataIndex));
                   ^
1 error generated.
make: *** [Release/obj.target/cpufeatures/src/binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/lib/build.js:203:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command \"/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node\" \"/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
gyp ERR! node -v v21.6.1
gyp ERR! node-gyp -v v9.4.1
gyp ERR! not ok"
info This module is OPTIONAL, you can safely ignore this error

[matthewkeil]

In the error output above one of the native dependencies needs to be built but you are missing python

/bin/sh: python: command not found

Make sure python is on your path and will likely build after that. We are working on getting you into discord now

[matthewkeil]

What is your discord handle @FaybianB ?

[jeluard]

That's the culprint: /bin/sh: python: command not found.

Before trying to have python detected, could you give this branch jeluard/upgrade-blst a shot?

[nflaig]

Before trying to have python detected, could you give this branch a shot?

~~Branch doesn't exist, we merged that change into unstable 3 weeks ago (see https://github.com/ChainSafe/lodestar/pull/6413)~~

https://github.com/ChainSafe/lodestar/tree/jeluard/upgrade-blst <-- correct ref to branch

[jeluard]

My bad, wrong use of markdown :)

[FaybianB]

/bin/sh: python: command not found

@matthewkeil So I do have python installed but it's python3. /usr/bin/python does not exist but usr/bin/python3 does. I've tried setting up an alias a few ways:

• echo "alias python=/usr/bin/python3" >> ~/.zshrc
• alias python=python3

and although these work in the terminal:

The yarn install fails with the same /bin/sh: python: command not found error. I've spent hours trying to fix this online but I'm stuck.

What is your discord handle @FaybianB ?

ugongetthisword

That's the culprint: /bin/sh: python: command not found.

Before trying to have python detected, could you give this branch jeluard/upgrade-blst a shot?

@jeluard @nflaig Before trying the Python fix above, I checked out this branch, ran yarn install and got the same result.

[jeluard]

You might want to give a shot at https://github.com/ChainSafe/bls/issues/148#issuecomment-1877328596 then. Or https://github.com/ChainSafe/bls/issues/148#issuecomment-1877330565

[jeluard]

@FaybianB what is your node version btw?

[jeluard]

Ok looks like you are using node 21. Can you switch to node 20 and give another shot at branch jeluard/upgrade-blst ?

[FaybianB]

You might want to give a shot at ChainSafe/bls#148 (comment) then. Or ChainSafe/bls#148 (comment)

The first technique yielded some results. yarn install still outputs some errors but it seems to overall have succeeded since subsequent yarn install commands says it's up to date:

ugongetthisword@Workss-MacBook-Pro lodestar % yarn install             
yarn install v1.22.21
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
[3/5] 🚚  Fetching packages...
[4/5] 🔗  Linking dependencies...
[5/5] 🔨  Building fresh packages...
[11/18] ⡀ @chainsafe/blst
[-/18] ⡀ waiting...
[12/18] ⡀ bcrypto
[14/18] ⡀ cpu-features
warning Error running install script for optional dependency: "/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features: Command failed.
Exit code: 1
Command: node buildcheck.js > buildcheck.gypi && node-gyp rebuild
Arguments: 
Directory: /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
Output:
gyp info it worked if it ends with ok
gyp info using [email protected]
gyp info using [email protected] | darwin | arm64
gyp info find Python using Python version 3.12.1 found at \"/opt/homebrew/opt/[email protected]/bin/python3.12\"
gyp info spawn /opt/homebrew/opt/[email protected]/bin/python3.12
gyp info spawn args [
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/gyp/gyp_main.py',
gyp info spawn args   'binding.gyp',
gyp info spawn args   '-f',
gyp info spawn args   'make',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features/build/config.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/addon.gypi',
gyp info spawn args   '-I',
gyp info spawn args   '/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/common.gypi',
gyp info spawn args   '-Dlibrary=shared_library',
gyp info spawn args   '-Dvisibility=default',
gyp info spawn args   '-Dnode_root_dir=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1',
gyp info spawn args   '-Dnode_gyp_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp',
gyp info spawn args   '-Dnode_lib_file=/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/<(target_arch)/node.lib',
gyp info spawn args   '-Dmodule_root_dir=/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features',
gyp info spawn args   '-Dnode_engine=v8',
gyp info spawn args   '--depth=.',
gyp info spawn args   '--no-parallel',
gyp info spawn args   '--generator-output',
gyp info spawn args   'build',
gyp info spawn args   '-Goutput_dir=.'
gyp info spawn args ]
gyp info spawn make
gyp info spawn args [ 'BUILDTYPE=Release', '-C', 'build' ]
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_aarch64_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_arm_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_mips_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_ppc_linux.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_freebsd.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_linux_or_android.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_macos.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/impl_x86_windows.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/filesystem.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/stack_line_reader.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/string_view.o
  CC(target) Release/obj.target/cpu_features/deps/cpu_features/src/hwcaps.o
  LIBTOOL-STATIC Release/cpu_features.a
  CXX(target) Release/obj.target/cpufeatures/src/binding.o
In file included from ../src/binding.cc:1:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/node.h:73:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8.h:24:
In file included from /Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-array-buffer.h:12:
/Users/ugongetthisword/Library/Caches/node-gyp/21.6.1/include/node/v8-local-handle.h:253:5: error: static_assert failed due to requirement 'std::is_base_of<v8::Value, v8::Data>::value' \"type check\"
    static_assert(std::is_base_of<T, S>::value, \"type check\");
    ^             ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
../../nan/nan_callbacks_12_inl.h:175:20: note: in instantiation of function template specialization 'v8::Local<v8::Value>::Local<v8::Data>' requested here
      cbinfo(info, obj->GetInternalField(kDataIndex));
                   ^
1 error generated.
make: *** [Release/obj.target/cpufeatures/src/binding.o] Error 1
gyp ERR! build error 
gyp ERR! stack Error: `make` failed with exit code: 2
gyp ERR! stack     at ChildProcess.onExit (/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/node-gyp/lib/build.js:203:23)
gyp ERR! stack     at ChildProcess.emit (node:events:519:28)
gyp ERR! stack     at ChildProcess._handle.onexit (node:internal/child_process:294:12)
gyp ERR! System Darwin 22.5.0
gyp ERR! command \"/Users/ugongetthisword/.nvm/versions/node/v21.6.1/bin/node\" \"/Users/ugongetthisword/Miscellaneous/lodestar/node_modules/.bin/node-gyp\" \"rebuild\"
gyp ERR! cwd /Users/ugongetthisword/Miscellaneous/lodestar/node_modules/cpu-features
✨  Done in 13.67s.






ugongetthisword@Workss-MacBook-Pro lodestar % yarn install
yarn install v1.22.21
[1/5] 🔍  Validating package.json...
[2/5] 🔍  Resolving packages...
success Already up-to-date.
✨  Done in 0.21s.

[FaybianB]

Ok looks like you are using node 21. Can you switch to node 20 and give another shot at branch jeluard/upgrade-blst ?

@jeluard That worked.

[jeluard]

Thanks for testing!

[FaybianB]

Back on my branch for this ticket:

• yarn install seems to be passing
• yarn build succeeds
• ./lodestar --help successfully displays the help
• ./scripts/run_e2e_env.sh start continues to timeout. Tried 3 times in a row.
• yarn test:e2e still fails, assuming because the above fails.

[matthewkeil]

/bin/sh: python: command not found

@matthewkeil So I do have python installed but it's python3. /usr/bin/python does not exist but usr/bin/python3 does. I've tried setting up an alias a few ways:

• echo "alias python=/usr/bin/python3" >> ~/.zshrc
• alias python=python3

and although these work in the terminal:

The `yarn install` fails with the same `/bin/sh: python: command not found` error. I've spent hours trying to fix this online but I'm stuck.

Unfortunately one of the dependencies of one of our dependencies has a hard coded (in an internal script) call to python and that gets run in a subshell that does not load dotfiles which is why you are seeing the alias work on the command line but not during installation. The only way around this is to use pyenv as it puts itself natively on PATH and then proxies the call to the appropriate python version (replacing python for python3 if necessary).

[jeluard]

@matthewkeil That part is fixed now @FaybianB You need to have docker running for ./scripts/run_e2e_env.sh start to work

[matthewkeil]

@FaybianB I sent a friend request to you on Discord. I will be able to manually add you to the server from there. Please feel free to DM me when you see it and we will get you settled in. Welcome!

[FaybianB]

@matthewkeil That part is fixed now @FaybianB You need to have docker running for ./scripts/run_e2e_env.sh start to work

So do I need to do that in order to run the end-to-end tests?

[jeluard]

You need to have docker running, yes.