Implement chain events

[elmattic]

Issue summary

Currently PR #4691 is blocked by the lack of events in Forest.

Task summary

• [ ] Propose a storage solution for events (i.e., that could be another column in paritydb). On Lotus side, a relational database is used (SQLite).
• [ ] Is it expected for our users to interact directly with this DB outside of Forest? What is the expected size of the DB when running Lotus on mainnet? What are the expected performance characteristics (see last point).
• [ ] Implement a minimal feature set (EventIndex, CollecteEvent types); we don't necessarily need to support all the features from the git-go, only what's required to unblock the above-mentioned PR.
• [ ] Figure out if we need to implement a similar DB migration scheme to Lotus (that's very unlikely as their storage choice seems implementation-specific). Using a similar relational DB, we could use their most up-to-date schema.
• [ ] Do we want to make historical events optional, and how? (if we still want to serve RPC requests, determine the endpoints that won't be supported). For example, a node without RPC enabled doesn't need this feature.
• [ ] Determine if we need support for actor events? (usually, yes, since we're interested in EVM actor logs)
• [ ] Implement rigorous testing
• [ ] Understand performance characteristics that we need to support, depending on:
  • Frequency of events
  • Mean/max number of filters used by node operators

Feel free to break down the issue into new tasks if needed, i.e., a research spike, etc..

Other information and links

Lotus events implementation: https://github.com/filecoin-project/lotus/tree/master/chain/events

Related FIPs:

• https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0049.md
• https://github.com/filecoin-project/FIPs/blob/master/FIPS/fip-0083.md

[elmattic]

Here's a non-exhaustive list of methods that are likely blocked by this issue:

• Filecoin.EthGetTransactionReceipt
• Filecoin.EthGetTransactionReceiptLimited
• Filecoin.EthGetFilterChange
• Filecoin.EthGetFilterLogs
• Filecoin.EthGetLogs
• ~~Filecoin.EthNewPendingTransactionFilter~~

[LesnyRumcajs]

@elmattic is EthNewPendingTransactionFilter blocked? There's a PR for it https://github.com/ChainSafe/forest/pull/4725. Or does it lack something?

[elmattic]

@elmattic is EthNewPendingTransactionFilter blocked? There's a PR for it #4725. Or does it lack something?

We could probably remove this one for now. We should be able to create a new filter even without directly supporting events.

Testing this endpoint properly would require something like Filecoin.EthGetFilterChange, which is on my list.

@sudo-shashank What do you think?

[sudo-shashank]

@elmattic is EthNewPendingTransactionFilter blocked? There's a PR for it #4725. Or does it lack something?

We could probably remove this one for now. We should be able to create a new filter even without directly supporting events.

Testing this endpoint properly would require something like Filecoin.EthGetFilterChange, which is on my list.

@sudo-shashank What do you think?

yes the list of methods shared above is blocked on the points mentioned in this issue

[LesnyRumcajs]

@elmattic Does it also block these methods? Any others?

• https://github.com/ChainSafe/forest/issues/4750
• https://github.com/ChainSafe/forest/issues/4751

[elmattic]

Does it also block these methods?

Indeed.

[elmattic]

Any others?

SubscribeActorEventsRaw

[elmattic]

EthSubscribe as well.

[ansermino]

@elmattic Can we improve the visibility of progress on this issue? Would be great to include a list of tasks (issues) that are required

It seems like a lot of open question potentially remain. Would it make sense to invest a bit more in the design phase before continuing with implementation?

[elmattic]

I propose closing this issue since the initial investigation is complete. We opted for a dedicated event cache, similar to the approach used in the Reth client.

Once the entire ETH-RPC API is implemented and performance tests are conducted (e.g. using lotus-bench), we can decide on the path forward, whether that involves implementing a dedicated indexer sidecar app or another solution.