kadena-io
Distinct "sync phase" upon startup
Currently, all our components start at the same time. Futher, the sync components assume that a node joining the network is already fairly close to the top of the Chainweb. This is an issue for a fresh node, who may have no data from the main "consensus line" at all. While syncing to try and catch up, this node will also be mining on the lowest block heights. This is completely wasted work.
Instead, we should add a preliminary phase where a node will do nothing but sync and verify Pact transactions. Once it gets "close enough" to the head of the Chainweb, it can resume normal operation as currently defined.
This preliminary phase can be it's own module that just calls existing functions. "New code" doesn't need to be written, per se. The call to this new module should occur before [this block of code]. Notice the "FIXME" ;)
[this block of code] https://github.com/kadena-io/chainweb-node/blob/master/src/Chainweb/Chainweb.hs#L424-L434
➤ lars commented:
We also have to do block header validation. Most of it is implied (i.e. checked) by inserting into the respective block header db. But adjacent parent verification isn’t. Currently it is done by recursively checking that all dependencies are available.
For an iterative approach one would have to ensure that blocks are added in the right order, so that all dependencies are added before the block itself.
I think there are two options:
- Use the current recursive approach and just switch off/leave off mining (and possibly cut processing).
- Use the (currently switched off) single-chain sync session to get all block headers (without parent validation and pact validation) and do a second path over all block in the db to run pact validation and adjacent parent validation.
The second is a little more tricky and possibly not worth it, because compared to a final solution, it’s just something like 50% feature wise but 90% in terms of dev costs, I guess.
With the first option there is a chance that we need some memory for storing the open tasks in the task queues during the recursive decent. There may even be some stack-build up (not sure, I would have to check the code). Still I propose to try that first and see how it performs.
➤ lars commented:
So, a good way to start would be to run Chainweb.Chainweb with mining and cut-processing disabled and switch both components on only when we synced with the head of Chainweb.
I am not sure how much overhead cut processing causes. So we may first try to just leave the miner disabled, which is easier to do.
➤ lars commented:
For turning off cut processing would would have to add a flag to the cut-Db queue processor to just drop new incoming cuts. However, we would still need to query a "starting" cut to start with. So it's most likely the easiest to leave cut-processing on.
➤ Colin Woodbury commented:
This PR https://github.com/kadena-io/chainweb-node/pull/92 addresses this task. It may already work as intended and need no further tweaks. I still need to test it a bit more.