go-ethereum icon indicating copy to clipboard operation
go-ethereum copied to clipboard

log indexer issue with `debug_setHead`

Open jwasinger opened this issue 6 months ago • 3 comments

I'm not yet sure if this is an actual error, or some spurious messages. Discovered this while looking into another issue with setHead in dev mode.

System information

OSX, Geth master branch

Expected behaviour

When executing dev mode, creating some blocks and calling debug_setHead to rewind the chain should work properly.

Actual behaviour

Some errors are being printed which originate from the log indexer:

ERROR[05-29|20:49:42.571] Header not found                         number=5 hash=135f17..90f6c6
ERROR[05-29|20:49:42.571] Header not found                         number=5 hash=135f17..90f6c6
ERROR[05-29|20:49:42.571] Header not found                         number=5 hash=135f17..90f6c6
ERROR[05-29|20:49:42.571] Header not found                         number=5 hash=135f17..90f6c6

jwasinger avatar May 29 '25 12:05 jwasinger

On it

ForrestKim42 avatar May 29 '25 17:05 ForrestKim42

It is just a spurious messages

ForrestKim42 avatar May 29 '25 18:05 ForrestKim42

Root Cause Analysis

This "Header not found" error is indeed a spurious message that occurs due to the asynchronous nature of geth's architecture during debug_setHead operations. Here's why:

Why It's Spurious

  1. Race Condition by Design: When debug_setHead is called, the blockchain immediately deletes headers above the specified block height. However, FilterMaps operates asynchronously and may still reference the old chain state.

  2. Temporary Inconsistency: The error occurs when ChainView.extendNonCanonical() tries to traverse backwards using stale hash references that point to already-deleted headers.

  3. Self-Recovery: FilterMaps automatically recovers when it processes the ChainHeadEvent and calls checkRevertRange(), which detects the inconsistency and reverts to a consistent state.

  4. No Functional Impact: Log indexing continues to work correctly despite these temporary error messages.

Proposed Fix

Change the log level from ERROR to DEBUG in core/filtermaps/chain_view.go:

if header == nil {
    log.Debug("Header not found during chain view extension", "number", number, "hash", hash)
    return false
}

This reflects the true nature of the message - it's an expected transient state during chain reorganization, not an actual error requiring user attention.

ForrestKim42 avatar May 29 '25 18:05 ForrestKim42