ethereum-optimism
Holocene-D: Action Tests
Things we would like from action testing:
- They should be easy to read, almost like a DSL with minimal boilerplate
Coverage we would like for the Holocene derivation:
- The fake batcher: some high level controls such as
- "Build channels x,y,z with frames foo,bar, submitted in a certain order on L1"
- control which frame goes into which L1 tx
- also out-of order batches in a single channel
- In frame queue unit tests, we can assert on the state of the derivation pipeline, but in e2e tests are assertions are probably limited to "the safe chain progressed or did not progress"
Also partial span batch validity should be tested.
The order of the batches should be correct, but midway through there should be an invalid batch.
We can have an invalid batch (make the parent hash wrong, or timestamp is incorrect https://github.com/ethereum-optimism/optimism/blob/13839ef701672a6965f1a34bd538af8008adbc9a/op-node/rollup/derive/batches.go#L34-L35) or a batch which results in an invalid payload. For an invalid payload (running out of gas or incorrect / reused nonce), the Holocene DP will generate an empty payload / deposit only block.
Pre holocene, the entire span batch becomes invalid.
We should concentrate on invalidation modes that changed in Holocene.
Post holocene,
- for an invalid batch only the part of the span batch after the bad block is invalid. So we would expect partial safe head progression (up to the block immediately prior to the bad one).
- for an invalid payload, we expect partial safe head progression up to and including the block number of the bad block. Only this time the block is replaced by a deposit only block (the entire block is discarded if any transaction is invalid e.g. has the wrong nonce). https://specs.optimism.io/protocol/holocene/derivation.html#engine-queue
We should be ready soon to run the Holocene assertions. We can stub them out until we are ready.
We also need to write tests to cover the Holocene activation itself (special case where things are reset).
We could submit the first frame (and ingest it) of a channel before the activation time, then submit the final frame after the activation. With Holocene we expect such a channel to be dropped / not derived from.
We should also test the new channel size limit (https://github.com/ethereum-optimism/specs/pull/431).
Reopening to track some more coverage we would like to add.
From @clabby: Some other tests that would be great to follow up on:
- Fault prove the holocene activation block, as well as holocene activation block + 1, for added coverage over the initial behavior of the payload building code.
- Coverage over the new channel size restriction in the channel stage, where compressed bytes must be < the max rlp bytes per channel protocol constant.
- @sebastianst's described edge case where we did need the frame ordering restriction in the actual Channel datatype, and not just the frame queue.
On the channel size limit. So far I have been able to write a test which constructs a >100MB channel, which with compression ON we can get on chain and trigger the expected behaviours (which are the same pre and post holocene). However it is quite a heavy / slow test (takes 28s on my laptop).
Furthermore, to test the different behaviour with Holocene we actually need a channel which is >100MB when compressed. I can create such a channel in my test by turning compression OFF -- but it is not possible to get the channel on chain with the current DA capacity. It will always time out and be rejected for that reason.
It might be possible to use alt-DA to do this, but I hesitate to go down that path (using a beta feature to test a core consensus rule).
One way forward is to define a custom test chain specification which lowers the MAX_RLP_BYTES_PER_CHANNEL parameter to something very low (e.g. 10KB). The test can then run very quickly and avoid channel timeout and L1 DA limits. But, we would need to do that for op-program and kona.
It is worth noting that we already have unit test coverage for the Go implementation for this test scenario https://github.com/ethereum-optimism/optimism/blob/38ee0891bab0c5c6607354eb3479d8597b808f4f/op-node/rollup/derive/channel_assembler_test.go#L95-L103
