hop
hop copied to clipboard
hop-protocol
Update `getLogs` batch size based on RPC provider
Different RPC providers allow for different getLogs lookback amounts on different networks. The hop-node should support the max lookback for different popular RPC providers. Some RPC providers don't have an explicit limit but rather just truncate the return data if there is too much, so the current DefaultBatchBlocks seems ideal for those cases.
Stats as of April 20, 2022:
- Alchemy
- No explicit limit on any chain, just data truncation
- Infura
- Mainnet - No explicit limit but a timeout if the call takes more than 10s
- Response:
{"jsonrpc":"2.0","id":1,"error":{"code":-32005,"message":"query timeout exceeded"} - Polygon - Max of
3500 - Arbitrum - Max of
100000 - Optimism - No explicit limit and no timeout. Seemingly no truncated data either 🤷
- Quiknode
- Gnosis chain - Max of
10000
- Gnosis chain - Max of
My current suggestion would be to have a dictionary of max lookback amounts per chain per provider, as I don't think there is a way to automate that and I don't believe that number changes often. With that said, I'm not sure what the best way to programmatically know what provider is being used.
An alternative suggestion would be to use a low but inefficient constant for all chains/providers (i.e. 3000), but that could get too resource intensive (RPC provider cost).
I'm not sure what the best way to programmatically know what provider is being used
since we have a proxy, we could add a custom path that returns the provider being used. For example, GET https://mainnet.rpc.example.com/upstream-host -> alchemy.com
That is a great idea & I think it makes a lot of sense. I think some of the stats listed above have since changed and keeping up-to-date with different providers might prove to be difficult.
With that said, I think the proposed solution (upstream-host & local dictionary) would work great when the bandwidth opens up to implement this.
the custom path to query provider name has been added to proxies
example
$ curl "https://arbitrum.rpc.example.com/upstream-provider"
{"provider": "alchemy"}
$ curl "https://gnosis.rpc.example.com/upstream-provider"
{"provider": "quiknode"}