go-ethereum
go-ethereum copied to clipboard
ethereum
all: live chain-aware tracing
This PR lets users run a long-running tracer. The tracer will run alongside normal chain processing operations. It will be aware of practically all observable changes to the chain. This allows people to write complete indexing solutions without needing a fork of geth with large footprint over sensitive and often-changing parts of the codebase. E.g. see the discussion at https://github.com/ethereum/go-ethereum/issues/27003. There are 2 major themes to this:
- Configuring a tracer to run as part of normal blockchain operations
- Extending the tracing interface to include high-level events such as
BlockStart,BlockEnd, as well as minute details likeOnNewAccount,OnBalanceChange, etc.
Given that the native tracers have access to the filesystem, it will be possible for example already to do a full sync and store all call traces in a file. However I plan to work on streamlining this and offering db persistence of traces directly in geth in the future. Then we can offer trace_filter-like APIs.
Backwards-compatibility
There has been a change in the hooks CaptureTxStart/CaptureTxEnd and CaptureStart/End/Enter/Exit. These hooks have been moved up slightly and will now capture tx/call validation errors. I.e. There will be e.g. CaptureEnter invocations even if the call fails validation (depth-check or value balance check), and consequently CaptureExit will be called with the error.
Greeting @s1na , I noticed that the StateLogger part of my pull request (https://github.com/ethereum/go-ethereum/pull/27617) aligns perfectly with what you have already implemented. I'm considering building upon your work. I have a few questions:
-
Currently, it seems that only logs are output when onXXX is triggered. Is it appropriate to add a subscription mode here (
event/feed/send())? -
I'm unsure if most nodes will enable the tracer module in the same way they enable the current subscription mode.
-
I'd like to clarify that my planned extension for the subscription mode is designed through difflayer. Regarding the onXXXX tracer, my understanding is that it retrieves the existing transactions. When it comes to adding logs, would it be better to adding subscription functionality through the existing difflayer module directly (which overlaps a bit with question 1)?
Hey @joohhnnn. That is an interesting idea to push tracing notifications on a channel to be used with a subscription method. We could have something like eth_subscribe('traces', { tracer: 'prestateTracer' }). Then add a way for tracers to push notifications. However we have to figure out how to detect in FilterSystem which subscription a given trace belongs to.
Edit: We can totally simplify this at the cost of some flexibility. eth_subscribe('traces') would subscribe to events coming out of the live tracer, with no option to choose different tracers. I think this is a good direction.
@s1na Thanks for this initial PR, I'm going to start build up upon it to see how it looks like and report once I have played with it a little bit more.
So I noticed an issue in the new hooks interface. We pass in the full transaction object in CaptureTxStart. I had adopted the prestateTracer to make use of this. It fetches the prestate for from, to in the beginning. Problem is when doing a debug_traceCall we don't have a valid signature in the tx object we simulate. Basically there is no good way to get the sender address for non-signed txes. So I added the from address as a separate field.
Note: if the tracer panics it will take down the node with it. We can add logic to recover from any tracer hook failure. But IMO this is a very privileged way of running the node and I expect only a few power users to do this. And they'd probably want to find out as soon as possible that something is wrong in their pipeline instead of missing events.
Needs a rebase again. If it were me, I'd probably squash this to one or two commits, it's easier to rebase a single commit than a sequence of 88 commits.
