`eth_getLogs` is very slow for v1.15.11

[wafeishushu]

trafficstars

After upgrading geth to v1.15.11, i found eth_getLogs rpc is very slow. You guys can try this curl to reproduce:

curl --location 'http://localhost:8545' \
--header 'Content-Type: application/json' \
--data '{
    "jsonrpc": "2.0",
    "method": "eth_getLogs",
    "params": [
        {
            "topics": [
                [
                    "0x25308c93ceeed162da955b3f7ce3e3f93606579e40fb92029faa9efe27545983",
                    "0x1df3f2a973a00d6635911755c260704e95e8a5876997546798770f76396fda4d",
                    "0x501781209a1f8899323b96b4ef08b168df93e0a90c673d1e4cce39366cb62f9b"
                ]
            ],
            "fromBlock": "0x1562cfd",
            "toBlock": "0x1562cfd",
            "address": [
                "0x2a3dd3eb832af982ec71669e178424b10dca2ede"
            ]
        }
    ],
    "id": 9723729
}'

[rjl493456442]

Please check if the logs are still indexing.

In v1.15.6, we shipped the new version log indexer and it takes a while (a few hours) to complete the indexing. You can check Log index head* logs for the progress. After that, eth_getLogs should be faster than before.

[s1na]

@wafeishushu Is this still an issue?

[kaber2]

I'm experiencing the same issue.

Yesterday I processed the block range from around 12,000,000 to head multiple times, with history.logs unset (so I assume 90k). There are around 25 parallel instances, each processing ranges of 2k blocks split over 8 goroutines each. Each filter is for 1 event, no addresses specified. The average speed was around 5k blocks/sec. Additionally the block headers for every block with relevant events are fetched.

I then changed history.logs=0 and let the complete index rebuild, everything else is unchanged. After this change, I see average speeds of around 2.3k blocks/sec.

I'll unindex the logs again to verify and will report the result.

[kaber2]

I discovered the history.logs.disable option and enabled it, and speed is back up to around 5k blocks/sec.

[kaber2]

One final note, in both cases I'm using cache.blocklogs=2560 to make sure the 2k block ranges fit into the log cache.

[zsfelfoldi]

Yesterday I processed the block range from around 12,000,000 to head multiple times, with history.logs unset (so I assume 90k).

The default indexed history for both transactions and logs is 2.350.000 blocks (cca 1 year). Querying anything older than that is expected to be slow. The issue in question though looks like a mutex handling bug that has already been fixed. With this issue fixed, once the indexing has been finished, querying in the indexed region should have a performance between 1M-100M blocks/sec depending on the complexity of the query (unless of course the query returns a lot of actual matches, in which case it can be slower).

[zsfelfoldi]

The original issue report lacks details but since it's a single block query for a recent block, it looks like a case of the mutex bug that has been fixed in v1.16.0: https://github.com/ethereum/go-ethereum/commit/91900e79eac9c5467a7b9effdd5e53dcf96943a8 I am closing this now, if the issue persists then please reopen and provide more information (--history.logs setting, has it finished chain syncing and indexing already, how slow it is).