[EPIC] Separate SignedHeader and Data into 2 different blobs

[nashqueue]

The PFB Commitment in the Header will reference the PFB commitment of the Data. This is only possible if it is a separate blob.

• [ ] When submitting a block, submit the header and Data separately
• [ ] One PFB that submits 2 blobs
• [ ] Match Header and Blob after they have been submitted separately
• [ ] Separate into 2 different namespaces
• [ ] https://github.com/rollkit/rollkit/issues/1828
• [x] https://github.com/rollkit/rollkit/issues/913

Testcases:

• [ ] Only Header is read but corresponding Block is not posted
• [ ] Only Block is read but corresponding Header is not posted

[Manav-Aggarwal]

Note: We can use something like Brotli compression for the Rollup block data part, but we shouldn't do it for headers so that rollup light nodes don't have to de-compress when downloading headers from the DA layer directly.

[Manav-Aggarwal]

Related: https://github.com/rollkit/rollkit/issues/1271, https://github.com/rollkit/rollkit/issues/772

[gupadhyaya]

while separating header and data parts of a block mainly for DA purposes, i came across a dilemma whether to keep the Block structure and corresponding storage (https://github.com/rollkit/rollkit/blob/main/store/types.go#L20) or modify it have Header and Data separately. There are two options here:

1. sequencer submits transactions which then gets processed by the block builder who submits block header to DA and creates a Block for storing and p2p gossip purposes. This will retain the current Block representation and storage. This option will not require hard-fork if there already exists rollkit rollups using the older store.
2. sequencer submits txs which then gets processed by the block builder which submits header to DA, stores Header and Data separately. This will require hard-fork to handle existing rollups with older storage formats and also we will have to modify the block sync to sync only txs (Data) and not whole Block.

both ways we could make it work. any preferences? @MSevey @Manav-Aggarwal @tzdybal

note that, if we don't have any rollups on mainnet yet, we can do 2 with clean store without requiring hardfork.

[gupadhyaya]

We may have to remove the P2P blocksync because with the separation, there won't be a notion of Block. Only SignedHeader and Data exists. While SignedHeader can still use p2p header sync as it respects the Header interface (https://github.com/celestiaorg/go-header/blob/main/header.go#L11), Data does not. I think for Data we can rely on syncing from the sequencer. All nodes (block builder or full nodes) will have sequencer grpc client, using which they can repeatedly call GetNextBatch to sync from their last Data to latest Data. May be some interface methods like GetBulkBatch could be added for fast sync. @Manav-Aggarwal @tzdybal what do you guys think about removing the p2p block sync and only relying on the sequencer?

[Manav-Aggarwal]

while separating header and data parts of a block mainly for DA purposes, i came across a dilemma whether to keep the Block structure and corresponding storage (https://github.com/rollkit/rollkit/blob/main/store/types.go#L20) or modify it have Header and Data separately. There are two options here:

1. sequencer submits transactions which then gets processed by the block builder who submits block header to DA and creates a Block for storing and p2p gossip purposes. This will retain the current Block representation and storage. This option will not require hard-fork if there already exists rollkit rollups using the older store.
2. sequencer submits txs which then gets processed by the block builder which submits header to DA, stores Header and Data separately. This will require hard-fork to handle existing rollups with older storage formats and also we will have to modify the block sync to sync only txs (Data) and not whole Block.

both ways we could make it work. any preferences? @MSevey @Manav-Aggarwal @tzdybal

note that, if we don't have any rollups on mainnet yet, we can do 2 with clean store without requiring hardfork.

Preference after discussion was to go with option 2 since it is cleaner.