ethereumjs-monorepo icon indicating copy to clipboard operation
ethereumjs-monorepo copied to clipboard

Wrap DNS discovery lookup in try/catch.

Open SeaMonkey82 opened this issue 1 year ago • 7 comments

Client currently errors and exits on startup.

[10-29|08:14:16] ERROR Error starting client queryTxt ECONNREFUSED all.holesky.ethdisco.net
[10-29|08:14:19] ERROR Uncaught error: queryTxt ECONNREFUSED all.holesky.ethdisco.net
[10-29|08:14:19] ERROR Error: queryTxt ECONNREFUSED all.holesky.ethdisco.net
    at QueryReqWrap.onresolve [as oncomplete] (node:internal/dns/promises:251:17)
[10-29|08:14:19] INFO Caught interrupt signal. Obtaining client handle for clean shutdown...
[10-29|08:14:19] INFO (This might take a little longer if client not yet fully started)
[10-29|08:14:19] INFO Client did not start properly, exiting ...

acolytec3 on Discord:

Hmm, looks like we're not wrapping a dns discovery lookup in try/catch

SeaMonkey82 avatar Nov 03 '23 12:11 SeaMonkey82

Can you confirm if this is still an issue? I started our client on holesky with the --discDns flag, turned on debug logging specific for DNS and it was able to get peer candidates from the discDns tree. I'm wondering if the issue above occurred because the dns server may have temporarily been down?

acolytec3 avatar Jan 03 '24 20:01 acolytec3

npm run-script client:start -- --dataDir /mnt/holesky/prysm-ethereumjs --network holesky --saveReceipts --rpc --rpcPort=8583 --rpcEngine --rpcEnginePort=8583 --port=23504 --extIP "1.2.3.4" --jwt-secret=/mnt/holesky/jwtsecret

> @ethereumjs/[email protected] client:start
> npm run client:start:js -- --dataDir /mnt/holesky/prysm-ethereumjs --network holesky --saveReceipts --rpc --rpcPort=8583 --rpcEngine --rpcEnginePort=8583 --port=23504 --extIP 1.2.3.4 --jwt-secret=/mnt/holesky/jwtsecret


> @ethereumjs/[email protected] client:start:js
> npm run build && node dist/bin/cli.js --dataDir /mnt/holesky/prysm-ethereumjs --network holesky --saveReceipts --rpc --rpcPort=8583 --rpcEngine --rpcEnginePort=8583 --port=23504 --extIP 1.2.3.4 --jwt-secret=/mnt/holesky/jwtsecret


> @ethereumjs/[email protected] build
> npm run build:common && mkdir -p ./src/trustedSetup/ && cp -Rf ./src/trustedSetups ./dist/src/


> @ethereumjs/[email protected] build:common
> ../../config/cli/ts-build.sh

[Node build] Using tsconfig.prod.cjs.json
> tsc --build ./tsconfig.prod.cjs.json
[Node build] Working... DONE

Skipping ESM build (no config available).


[Post Build Fixes]Skipping post build fixes (no ESM setup yet).


[01-03|23:29:27] INFO Sync Mode full
[01-03|23:29:27] INFO Data directory: /mnt/holesky/prysm-ethereumjs
[01-03|23:29:27] INFO Full sync mode
[01-03|23:29:27] INFO Initializing Ethereumjs client version=v0.9.0 network=holesky chainId=17000
[01-03|23:29:27] INFO Preparing for sync using FullEthereumService with FullSynchronizer.
[01-03|23:29:27] INFO Latest local block number=178731 td=1 hash=0x88a5…5d0b hardfork=shanghai
[01-03|23:29:27] INFO Initializing VM merkle statemanager genesis hardfork=
[01-03|23:29:27] INFO Setting up merkleVM
[01-03|23:29:27] INFO Initializing account cache size=400000
[01-03|23:29:27] INFO Initializing storage cache size=200000
[01-03|23:29:27] INFO Initializing code cache size=200000
[01-03|23:29:27] INFO Initializing trie cache size=200000
[01-03|23:29:27] INFO Setup EVM execution.
[01-03|23:29:27] INFO Setup networking and services.
[01-03|23:29:27] INFO Started eth service.
[01-03|23:29:27] INFO Skipped execution run vmHead=178720 canonicalHead=178731 hardfork=shanghai execution=true
[01-03|23:29:27] INFO Started rlpx server maxPeers=25
[01-03|23:29:27] INFO Started discovery service discV4=true dns=true refreshInterval=30000
[01-03|23:29:27] INFO Server listener up transport=rlpx url=enode://55a545acc084410e204e6c066b999517772ce40873f08791c37936a737c8bb45d719ad3ed29905218f8b5665b089ff47b07cbeaac7e418be140cdd416e1dab5f@1.2.3.4:23504
[01-03|23:29:27] WARN Execution of block number=178721 hash=0x4624…2867 hardfork=shanghai failed:
Error: invalid RLP: remainder must be zero
[01-03|23:29:27] ERROR Error starting client queryTxt ECONNREFUSED all.holesky.ethdisco.net
[01-03|23:29:30] ERROR Uncaught error: queryTxt ECONNREFUSED all.holesky.ethdisco.net
[01-03|23:29:30] ERROR Error: queryTxt ECONNREFUSED all.holesky.ethdisco.net
    at QueryReqWrap.onresolve [as oncomplete] (node:internal/dns/promises:251:17)
[01-03|23:29:30] INFO Caught interrupt signal. Obtaining client handle for clean shutdown...
[01-03|23:29:30] INFO (This might take a little longer if client not yet fully started)
[01-03|23:29:30] INFO Client did not start properly, exiting ...

SeaMonkey82 avatar Jan 04 '24 04:01 SeaMonkey82

Have asked others from the team for input. When I use your startup parameters for ethereunjs, I'm not able to reproduce this error and my local node connects to peers obtained from the dns discovery service.

acolytec3 avatar Jan 08 '24 19:01 acolytec3

For privacy/security reasons, I force my LAN to use the local DNS server by blocking all outbound traffic on port 53. If you want to reproduce the issue, do that. Temporarily disabling this rule on the firewall, or running with --discDns=false, allows ethereumjs to run. All other clients are still able to function without DNS discovery, and ethereumjs used to be able to without explicitly disabling DNS discovery.

SeaMonkey82 avatar Jan 11 '24 13:01 SeaMonkey82

I see that -dnsAddr defaults to 8.8.8.8, which is what is actually being blocked and causing the client to fail. Is there a good reason to hardcode Google's DNS here instead of using the system default for DNS resolution?

Also, after supplying my own dnsAddr, I noticed this warning, which I don't recall seeing before.

[01-13|13:06:05] WARN Server error: Error - Number exceeds 53 bits

SeaMonkey82 avatar Jan 13 '24 17:01 SeaMonkey82

@SeaMonkey82 how can we access the systems default here?

Would that be:

const dns = require('dns');

dns.getServers();

?? (I am so so so not into this DNS stuff)

for me this is giving '192.168.178.1', which is basically just the internal network address??

Would this work?

holgerd77 avatar Jan 15 '24 10:01 holgerd77

You can confirm by running dig ethdisco.net from the command line and observing the SERVER that responds.

SeaMonkey82 avatar Jan 15 '24 16:01 SeaMonkey82