op-replica
op-replica copied to clipboard
optimisticben
Nonce too high error.
Starting today I suddenly keep getting this error when running my node.
l2geth-replica_1 | INFO [03-06|16:38:55.310] Syncing transaction range start=4175326 end=4177162 backend=l2
l2geth-replica_1 | DEBUG[03-06|16:38:55.311] Applying transaction to tip index=4175326 hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55 origin=sequencer
l2geth-replica_1 | DEBUG[03-06|16:38:55.312] Attempting to commit rollup transaction hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55
l2geth-replica_1 | ERROR[03-06|16:38:55.312] Problem committing transaction msg="nonce too high"
l2geth-replica_1 | ERROR[03-06|16:38:55.312] Got error waiting for transaction to be added to chain msg="nonce too high"
l2geth-replica_1 | ERROR[03-06|16:38:55.312] Could not verify error="Verifier cannot sync transactions with BackendL2: Verifier cannot sync transactions with backend l2: Cannot sync transactions with backend l2: Cannot apply transaction: nonce too high"
l2geth-replica_1 | DEBUG[03-06|16:38:55.755] Served eth_getBlockByNumber conn=172.19.0.3:59580 reqid=863 t=132.468µs
l2geth-replica_1 | INFO [03-06|16:38:56.310] Syncing transaction range start=4175326 end=4177163 backend=l2
l2geth-replica_1 | DEBUG[03-06|16:38:56.311] Applying transaction to tip index=4175326 hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55 origin=sequencer
l2geth-replica_1 | DEBUG[03-06|16:38:56.311] Attempting to commit rollup transaction hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55
l2geth-replica_1 | ERROR[03-06|16:38:56.312] Problem committing transaction msg="nonce too high"
l2geth-replica_1 | ERROR[03-06|16:38:56.312] Got error waiting for transaction to be added to chain msg="nonce too high"
l2geth-replica_1 | ERROR[03-06|16:38:56.312] Could not verify error="Verifier cannot sync transactions with BackendL2: Verifier cannot sync transactions with backend l2: Cannot sync transactions with backend l2: Cannot apply transaction: nonce too high"
l2geth-replica_1 | DEBUG[03-06|16:38:56.760] Served eth_getBlockByNumber conn=172.19.0.3:59582 reqid=864 t=171.777µs
l2geth-replica_1 | INFO [03-06|16:38:57.309] Syncing transaction range start=4175326 end=4177163 backend=l2
l2geth-replica_1 | DEBUG[03-06|16:38:57.311] Applying transaction to tip index=4175326 hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55 origin=sequencer
l2geth-replica_1 | DEBUG[03-06|16:38:57.311] Attempting to commit rollup transaction hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55
l2geth-replica_1 | ERROR[03-06|16:38:57.312] Problem committing transaction msg="nonce too high"
l2geth-replica_1 | ERROR[03-06|16:38:57.312] Got error waiting for transaction to be added to chain msg="nonce too high"
l2geth-replica_1 | ERROR[03-06|16:38:57.312] Could not verify error="Verifier cannot sync transactions with BackendL2: Verifier cannot sync transactions with backend l2: Cannot sync transactions with backend l2: Cannot apply transaction: nonce too high"
l2geth-replica_1 | DEBUG[03-06|16:38:57.764] Served eth_getBlockByNumber conn=172.19.0.3:59584 reqid=865 t=165.284µs
I am running an Optimism replica node, cloned op-replica and everything worked fine for weeks, until suddenly today when I rebooted, I got all these error. Anyone any idea what could have gone wrong?
Thanks for the report. Can you share the version of the data-transport-layer and l2geth containers? Also thens the containers are using 🙏
ethereumoptimism/replica-healthcheck
These were the versions I was running. Will now update to latest versions, and see if that will work.
Now updated to the latest versions:
REPOSITORY TAG IMAGE ID CREATED SIZE
ethereumoptimism/data-transport-layer latest ba9b21fc27a1 11 hours ago 839MB
ethereumoptimism/replica-healthcheck latest c4543115d5fa 11 hours ago 834MB
ethereumoptimism/l2geth latest 0706d9e8e03b 11 hours ago 43.8MB
Still same errors:
l2geth-replica_1 | DEBUG[03-07|18:32:47.454] Served eth_getBlockByNumber conn=172.23.0.4:50268 reqid=46 t=144.136µs
l2geth-replica_1 | INFO [03-07|18:32:47.638] Syncing transaction range start=4175326 end=4212315 backend=l2
l2geth-replica_1 | DEBUG[03-07|18:32:47.640] Applying transaction to tip index=4175326 hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55 origin=sequencer
l2geth-replica_1 | DEBUG[03-07|18:32:47.640] Attempting to commit rollup transaction hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55
l2geth-replica_1 | ERROR[03-07|18:32:47.640] Problem committing transaction msg="nonce too high"
l2geth-replica_1 | ERROR[03-07|18:32:47.640] Got error waiting for transaction to be added to chain msg="nonce too high"
l2geth-replica_1 | ERROR[03-07|18:32:47.640] Could not verify error="Verifier cannot sync transactions with BackendL2: Verifier cannot sync transactions with backend l2: Cannot sync transactions with backend l2: Cannot apply transaction: nonce too high"
l2geth-replica_1 | DEBUG[03-07|18:32:48.457] Served eth_getBlockByNumber conn=172.23.0.4:50270 reqid=47 t=165.786µs
l2geth-replica_1 | INFO [03-07|18:32:48.638] Syncing transaction range start=4175326 end=4212318 backend=l2
l2geth-replica_1 | DEBUG[03-07|18:32:48.640] Applying transaction to tip index=4175326 hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55 origin=sequencer
l2geth-replica_1 | DEBUG[03-07|18:32:48.640] Attempting to commit rollup transaction hash=0xd25d9918fe9d7927d6c14d41c5f925480c48dd985d0e3bb0377199caf226ee55
l2geth-replica_1 | ERROR[03-07|18:32:48.640] Problem committing transaction msg="nonce too high"
l2geth-replica_1 | ERROR[03-07|18:32:48.640] Got error waiting for transaction to be added to chain msg="nonce too high"
l2geth-replica_1 | ERROR[03-07|18:32:48.641] Could not verify error="Verifier cannot sync transactions with BackendL2: Verifier cannot sync transactions with backend l2: Cannot sync transactions with backend l2: Cannot apply transaction: nonce too high"
Did you update the l2geth instance for the Berlin hardfork? https://changelog.optimism.io/2022/02/15/mainnet-berlin-hardfork.html
from my l2geth-replica.env:
L2GETH_GENESIS_URL=https://storage.googleapis.com/optimism/mainnet/genesis-berlin.json
L2GETH_GENESIS_HASH=0x106b0a3247ca54714381b1109e82cc6b7e32fd79ae56fbcc2e7b1541122f84ea
L2GETH_BERLIN_ACTIVATION_HEIGHT=3950000
So yes would say it's updated right?
You have to geth init that file to activate it.
You can check like https://github.com/optimisticben/op-replica/blob/main/kustomize/replica/bases/configmaps/check-for-chaindata-berlin.sh#L25-L31
We updated the script, previously it only checked the chaindata directory existed, the updated script checks for the berlin activation height in the datadir. So, an old script (or none) would not automatically apply initialization.
optimism@ip:~/op-replica$ docker-compose logs -f l2geth-replica
Attaching to op-replica_l2geth-replica_1
l2geth-replica_1 | + echo running /scripts/check-for-chaindata-berlin.sh
l2geth-replica_1 | + GETH_DATA_DIR=/geth
l2geth-replica_1 | + GETH_CHAINDATA_DIR=/geth/geth/chaindata
l2geth-replica_1 | + GETH_KEYSTORE_DIR=/geth/keystore
l2geth-replica_1 | + '[' '!' -d /geth/keystore ]
l2geth-replica_1 | running /scripts/check-for-chaindata-berlin.sh
l2geth-replica_1 | + '[' '!' -d /geth/geth/chaindata ]
l2geth-replica_1 | + echo '/geth/geth/chaindata exists, checking for hardfork.'
l2geth-replica_1 | + echo 'Chain config:'
l2geth-replica_1 | + geth dump-chain-cfg '--datadir=/geth'
l2geth-replica_1 | /geth/geth/chaindata exists, checking for hardfork.
l2geth-replica_1 | Chain config:
l2geth-replica_1 | INFO [03-09|17:04:32.714] Maximum peer count ETH=50 LES=0 total=50
l2geth-replica_1 | INFO [03-09|17:04:32.714] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory"
l2geth-replica_1 | INFO [03-09|17:04:32.715] Allocated cache and file handles database=/geth/geth/chaindata cache=16.00MiB handles=16
l2geth-replica_1 | {
l2geth-replica_1 | "chainId": 10,
l2geth-replica_1 | "homesteadBlock": 0,
l2geth-replica_1 | "eip150Block": 0,
l2geth-replica_1 | "eip150Hash": "0x0000000000000000000000000000000000000000000000000000000000000000",
l2geth-replica_1 | "eip155Block": 0,
l2geth-replica_1 | "eip158Block": 0,
l2geth-replica_1 | "byzantiumBlock": 0,
l2geth-replica_1 | "constantinopleBlock": 0,
l2geth-replica_1 | "petersburgBlock": 0,
l2geth-replica_1 | "istanbulBlock": 0,
l2geth-replica_1 | "muirGlacierBlock": 0,
l2geth-replica_1 | "berlinBlock": 3950000,
l2geth-replica_1 | "clique": {
l2geth-replica_1 | "period": 0,
l2geth-replica_1 | "epoch": 30000
l2geth-replica_1 | }
l2geth-replica_1 | }
l2geth-replica_1 | + geth dump-chain-cfg '--datadir=/geth'
l2geth-replica_1 | + grep -q '"berlinBlock": 3950000'
l2geth-replica_1 | INFO [03-09|17:04:32.806] Maximum peer count ETH=50 LES=0 total=50
l2geth-replica_1 | INFO [03-09|17:04:32.806] Smartcard socket not found, disabling err="stat /run/pcscd/pcscd.comm: no such file or directory"
l2geth-replica_1 | INFO [03-09|17:04:32.807] Allocated cache and file handles database=/geth/geth/chaindata cache=16.00MiB handles=16
l2geth-replica_1 | Hardfork already activated.
l2geth-replica_1 | + echo 'Hardfork already activated.'
I did geth init and if I look here the hardfork is activated, but I still have the nonce errors.
Did the geth init run after the activation height 3950000?
If so, you may need to resync.