ape icon indicating copy to clipboard operation
ape copied to clipboard

Contract log fetching / querying is unnecessarily inefficient in multiple respects

Open fubuloubu opened this issue 3 years ago • 1 comments

When iterating over event logs, it is very slow to collect them together, because it tries to iterate over the entire blockchain's history, when the contract only deployed at some block within that history: https://github.com/ApeWorX/ape/blob/4f53967e4f3f2597b0d742ae4eb9cf1f5f15a264/src/ape/api/providers.py#L917-L934

  1. Event log start block should be bound to whenever the contract was first deployed, to the point it last existed (SELFDESTRUCT). If there is a change in contract code within using a CREATE2 proxy, those might make changes to end block as well
  2. How batching works can be extremely inefficient. Some blocks have no events, and some have a lot.
  3. Web3py provides a direct API for filtering events, some research should be done to determine if that might improve this section of code
  4. Further work should continue to be done on the query layer, which might vastly speed up the process of obtaining logs via data pipeline plugins

fubuloubu avatar Nov 02 '22 20:11 fubuloubu

Some elements were resolved in #1548, but further research is required if the above mentioned elements are still valid (and if there are more)

fubuloubu avatar Jul 25 '23 00:07 fubuloubu