truffle

truffle copied to clipboard

• I've opened a support ticket before filing this issue.

---

Issue

RPC endpoint errors causing nodejs process to crash. I am connecting to HDWalletProvider on a nodejs backend. It's typical to receive once in a while an error from RPC end point like time out or PollingBlockTracker. This is understandable and not the issue. The issue is that any RPC error cause my nodejs server to crash.

Is there a way to catch RPC errors to prevent crashing and handle it by skipping the block or retrying or whatever.

Please note that this issue is not about the timeout error or polling block tracker error. This issue is about the process crashing when those errors occur.

Steps to Reproduce

Simple initialization of web3 with RPC end point like this. const web3_56 = new Web3(new HDWalletProvider(process.env.PRIVATE_PRODUCTION, ${process.env.BSC_MAINNET_NODE}));//bscmainnet Then initializing contracts like this. If the contract has some events the errors are more likely to occur.

var contract = new web3.eth.Contract(abi, address);

Leave the process even without activity until some kind of RPC end point error occurs (timeout or polling block tracker or whatever).

Expected Behavior

I expect to have a way to catch those errors and decide in code what to do without having my nodejs process crash.

Actual Results

My nodejs process crash if any RPC error occurs.

C:\prj\server\node_modules\safe-event-emitter\index.js:74 throw err ^

Error: PollingBlockTracker - encountered an error while attempting to update latest block: undefined at PollingBlockTracker._performSync (C:\prj\server\node_modules\eth-block-tracker\src\p at runMicrotasks () Emitted 'error' event on Web3ProviderEngine instance at: at safeApply (C:\prj\server\node_modules\safe-event-emitter\index.js:70:5) at PollingBlockTracker.SafeEventEmitter.emit (C:\prj\server\node_modules\safe-event-emitter\index.js:56:5) at PollingBlockTracker._performSync (C:\prj\server\node_modules\eth-block-tracker\src\polling.js:53:16) at runMicrotasks () at processTicksAndRejections (internal/process/task_queues.js:95:5)

Environment

• Operating System:
• Windows
• Ethereum client:
• hdwallet-provider
• Truffle version:
• @truffle/hdwallet-provider": "^2.0.0
• node version (node --version):
• v14.18.2
• npm version (npm --version):
• 6.14.15

Thanks @stranger-games for raising this! I agree, we should definitely dig into this a bit to try and prevent this behavior. We'll have to do some investigating to try and figure out exactly what is going on. We'll put this in the backlog. Thanks again!

Hm, so here's a thought on how to deal with this...

Firstly, the naïve approach might be to add .catch() clauses around every use of this.initialized across the HDWallet code. Problem is: there's a race condition! Since this.initialized = this.initialize() gets invoked during the constructor, then awaited later, there's a slice of time where a rejection will be unhandled, which we all know that Node.js abhors. And we can't just put a .catch() in the constructor, since that's a synchronous context and the same sort of dragons exist.

So what's the alternative? We could change the behavior of this.initialize() so as never to reject for known failure modes. Instead, we make the promise returned by this.initialize() resolve to one of two kinds of values: one for success and one for error. Then, whenever we need to await this.initialized (like in send() or sendAsync()), we first check the resolved promise and handle the failure case with a proper in-line exception then and there. No unhandled rejections for Node.js to complain about!

After more investigation I'm not sure that this is going to be easily solvable. Looks like during initialization of web3-provider-engine it automatically tries to fetch block info in the background. This then causes it to fail if there is a bad connection/url. Since we don't have access to the internals of web3-provider-engine it seems to be out of our reach. :(

I encountered the same problem, when I tested how my project handles network errors. (I disabled the network before starting the project, so node exited because of the unhandled rejected promise). Since I need a solution now, I "fixed" the problem with a dirty workaround: I overwrote the initialize to never reject. Here is an example, if someone has the same problem:

const HDWalletProvider = require("@truffle/hdwallet-provider");

function sleep(millis) {
	return new Promise(resolve => setTimeout(resolve, millis));
}

async function run() {
	const origInit = HDWalletProvider.prototype.initialize;
	HDWalletProvider.prototype.initialize = async function () {
		while (true) {
			try {
				return await origInit.call(this);
			} catch (e) {
				console.log("origInit failed");
				console.log(e);
			}
			await sleep(1000);
		}
	};
	const provider = new HDWalletProvider({
		// private key is known in the web already
		privateKeys: ["8da4ef21b864d2cc526dbdb2a120bd2874c36c9d0a1fb7f8c63d7f7a8b41de8f"],
		providerOrUrl: "http://10.0.2.2:8545",
		addressIndex: 0,
	});
	provider.engine._blockTracker.on('error', function (e) {
		console.log('BlockTracker error', e);
		console.log(e);
	});
	provider.engine.on('error', function (e) {
		console.log('Web3ProviderEngine error');
		console.log(e);
	});
	await sleep(60 * 60 * 1000);
	provider.engine.stop();
}

run().catch(console.log);