node icon indicating copy to clipboard operation
node copied to clipboard

Node stays behind approximately distance to snapshot.

Open trufa opened this issue 1 year ago • 5 comments

I have installed I'm running a base client pointed towards my up to date, synced L1.

eth_blockNumber of my L2 stays behind for approximately the same distance to the end of the snapshots, and then keeps the pace but doesn't' advance.

For example if I point HOST_DATA_DIR to the /reth-data (empty) it starts syncing very fast but from the very beginning

If I point it to my snapshot it starts also syncing very fast

node-1 | t=2024-10-26T01:58:38+0000 lvl=info msg="Advancing bq origin" origin=0x42bccb52f4bcdfd7cb77145632f14f98e3c276c8b370d0322c5a7ec68d683231:21015340 originBehind=false

but until certain point and they kind of keeps up but always lagging behind around 3-4 days which is the distance to the end of my snapshotted data.

I've tried running it --l1.trustrpc but doesn't seem to make any difference.

The following command as suggested by the docs:

echo Latest synced block behind by: $((($( date +%s )-
$( curl -s -d '{"id":0,"jsonrpc":"2.0","method":"optimism_syncStatus"}' -H "Content-Type: application/json" http://localhost:7545 | jq -r .result.unsafe_l2.timestamp))/60)) minutes

Always keeps falling behind.

Any pointers would be greatly appreciated.

The errors I see in the log is one that I don't think is related

execution-1 | ts=2024-10-26T01:08:00.38459394Z level=error target=engine::jwt-validator message="Invalid JWT: Authorization header is missing or invalid"

And one that might be

node-1 | t=2024-10-26T01:51:23+0000 lvl=warn msg="Engine temporary error" err="temp: failed to sync forkchoice with engine: context deadline exceeded"

The peer count is in general around 8 and goes up to 11

trufa avatar Oct 26 '24 02:10 trufa

i was/am still hitting same thing , can you check log of reth if synchronising in batch mode?!

OP_NODE_SYNCMODE=execution-layer
OP_NODE_L2_ENGINE_KIND=reth <== seem without this  https://docs.optimism.io/builders/node-operators/configuration/consensus-config#l2enginekind reth doesn't go into batch mode for me, and it is missing in env example

and i also needed to tune this to be 10x larger https://reth.rs/run/config.html#merkle

then i can go to in-sync from similar scenario of few days, but still some performance issues unsolved occasionally(random beteween few seconds - 1hour delay some times ) on reth (already on nvme & not seeing obvious cpu bottleneck)

jun0tpyrc avatar Oct 29 '24 00:10 jun0tpyrc

@jun0tpyrc I honestly gave up trying to make reth work and now switched to geth, it seems to be syncing reasonably fast now.

trufa avatar Oct 29 '24 08:10 trufa

I had quite hard times here. I experienced static 12h drag and struggled a lot to fix it. OP_NODE_L2_ENGINE_KIND=reth fixes the problem for me. I believe it should be documented in this guide as there is no mention of it, not even in .env files

lacotWin avatar Oct 31 '24 08:10 lacotWin

I had quite hard times here. I experienced static 12h drag and struggled a lot to fix it. OP_NODE_L2_ENGINE_KIND=reth fixes the problem for me. I believe it should be documented in this guide as there is no mention of it, not even in .env files

wonder what spec you on &/ able to stay within minute of delay on reth for base-mainnet?! 🙏

jun0tpyrc avatar Nov 01 '24 11:11 jun0tpyrc

peer count 8-11 is quite low for an optimized performance, but if you managed to fix the issue great to hear ! true that OP_NODE_L2_ENGINE_KIND=reth is something you might have to add on configs, but its something you dont necessarily need but can be useful to know if you encounter some problems.

azeonmarket avatar Nov 02 '24 14:11 azeonmarket

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

github-actions[bot] avatar Nov 18 '25 00:11 github-actions[bot]

This issue was closed because it has been inactive for 5 days since being marked as stale.

github-actions[bot] avatar Nov 26 '25 00:11 github-actions[bot]