reth
reth copied to clipboard
feat(stages): log indexing
Description
This PR introduces log address and topic indexing - supplementary indices for log querying (intended to be used by RPC module).
Two new tables are added to the database:
LogAddressHistory- mapping of log address to block numbers where the logs were emitted by this addressLogTopicHistory- mapping of log topic to block numbers where this topic occurred.
The IndexLogHistoryStage walks over Receipts table and creates indices for log addresses and topics in corresponding tables.
Good start..
- Can we run this on an existing node to backfill via
reth stage run --commit? - How big are the tables on mainnet at tip?
@gakonst https://github.com/paradigmxyz/reth/pull/3089 solves this by starting the pipeline for any newly added stage
Codecov Report
Merging #3082 (c4131ca) into main (428a6dc) will decrease coverage by
0.09%. The diff coverage is48.81%.
| Impacted Files | Coverage Δ | |
|---|---|---|
| bin/reth/src/args/rpc_server_args.rs | 47.22% <0.00%> (-0.40%) |
:arrow_down: |
| bin/reth/src/args/stage_args.rs | 0.00% <ø> (ø) |
|
| bin/reth/src/stage/drop.rs | 2.00% <0.00%> (-0.13%) |
:arrow_down: |
| crates/consensus/common/src/validation.rs | 78.68% <0.00%> (-0.43%) |
:arrow_down: |
| crates/interfaces/src/test_utils/generators.rs | 98.15% <ø> (+7.36%) |
:arrow_up: |
| crates/primitives/src/lib.rs | 100.00% <ø> (ø) |
|
| crates/primitives/src/log.rs | 100.00% <ø> (ø) |
|
| crates/rpc/rpc-builder/src/auth.rs | 44.16% <0.00%> (-0.46%) |
:arrow_down: |
| crates/rpc/rpc-builder/src/lib.rs | 67.49% <0.00%> (-0.15%) |
:arrow_down: |
| crates/stages/src/sets.rs | 0.00% <0.00%> (ø) |
|
| ... and 20 more |
... and 7 files with indirect coverage changes
| Flag | Coverage Δ | |
|---|---|---|
| integration-tests | 15.96% <0.00%> (-0.13%) |
:arrow_down: |
| unit-tests | 63.89% <48.81%> (-0.06%) |
:arrow_down: |
Flags with carried forward coverage won't be shown. Click here to find out more.
| Components | Coverage Δ | |
|---|---|---|
| reth binary | 26.36% <0.00%> (-0.06%) |
:arrow_down: |
| blockchain tree | 81.25% <ø> (ø) |
|
| pipeline | 87.00% <90.11%> (+0.09%) |
:arrow_up: |
| storage (db) | 72.50% <34.38%> (-0.99%) |
:arrow_down: |
| trie | 94.66% <ø> (ø) |
|
| txpool | 49.11% <ø> (ø) |
|
| networking | 77.91% <ø> (+0.02%) |
:arrow_up: |
| rpc | 57.69% <6.14%> (-0.36%) |
:arrow_down: |
| consensus | 62.04% <0.00%> (-0.07%) |
:arrow_down: |
| revm | 34.90% <ø> (ø) |
|
| payload builder | 6.83% <ø> (ø) |
|
| primitives | 88.54% <100.00%> (+0.26%) |
:arrow_up: |
There's a conflict here now, sorry about that. Did we sync a mainnet node with this to sync how big the DB grew by it?
@gakonst We need to figure out when to release this - there is another breaking DB change in #3464, so I wonder if we should bundle both in e.g. Alpha 3?