chifra logs: Pretty easy speedup when querying logs

[tjayrush]

One user (thanks Matt!) pointed to a very common use case:

Give me all the events with topic emitted by address

He uses ethers.js and it returns the query very quickly (less than a minute for a very popular address). TrueBlocks takes hours when using chifra export --emitter <address>.

This is because TrueBlocks does not use the filtering function the node provides for eth_getLogs.

We kind of knew this path was slow for us, but have been ignoring it since we always query for full transactional details and/or traces, but it is a very clear primary use case for most people.

We should bite the bullet and implement eth_getLogs with filtering.

[tjayrush]

ethers.js code:

const pairCreatedAbi = ["event PairCreated(address indexed token0, address indexed token1, address pair, uint)"];
const provider = new JsonRpcProvider(process.env.ETH_RPC_URL);
const factory = new Contract(UNISWAP_V2_FACTORY_ADDRESS, pairCreatedAbi, provider);
const pairCreatedFilter = factory.filters.PairCreated();
const logs = await factory.queryFilter(pairCreatedFilter, 10000834, 10500000);
const parsedLogs = logs.map(factory.interface.parseLog);

10000834, 10500000 is the block range You can leave out the 10500000 input if you want to query until the current block

[tjayrush]

Version 2.0 of the Uniswap Factory: 0x5c69bee701ef814a2b6a3edd4b1652cb9cc5aa6f Version 3.0: 0x1f98431c8ad98523631ae4a59f267346ea31f984

[tjayrush]

Also, we should use eth_getBlockReceipts which is a new routine that was added a few years ago to some nodes but which we do not use.

[tjayrush]

@mds1 I started work on this issue, but I'm not very good with Javascript.

I installed ethers.js and tried to copy the above code into a file (test.js).

When I run it with node test.js it reports an error.

const logs = await factory.queryFilter(pairCreatedFilter, 10000834, 10500000);
             ^^^^^

SyntaxError: await is only valid in async functions and the top level bodies of modules
    at Object.compileFunction (node:vm:352:18)
    at wrapSafe (node:internal/modules/cjs/loader:1025:15)
    at Module._compile (node:internal/modules/cjs/loader:1059:27)
    at Object.Module._extensions..js (node:internal/modules/cjs/loader:1147:10)
    at Module.load (node:internal/modules/cjs/loader:975:32)
    at Function.Module._load (node:internal/modules/cjs/loader:822:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:81:12)
    at node:internal/main/run_main_module:17:47

Node.js v17.0.1

Would it be possible to provide a re-producable ethers.js script I can use to test performance? Plus, a tiny bit of "Here's how you run it..."

[dszlachta]

@tjayrush the error happens because it is invalid in current JS to put await outside of a function. Maybe you could try wrapping your code into a function that is called immediately:

(async function() {
  // the code
})()

[tjayrush]

I got it working. Thanks.

[tjayrush]

@dszlachta When you get around to do the getLogs port, perhaps we can look into this.

[tjayrush]

Copied this issue into a TODO item in the code. When we port to GoLang, we will find it.