Create the hard fork config automatically at the stop chain slot

[cjjdespres]

If all of the relevant hard fork parameters (the slot_tx_end, slot_chain_end, and hard_fork_genesis_delta) are all set, then the daemon will start a background thread that waits for the scheduled stop_chain_slot. Once there, it further waits until the consensus advances past the slot_tx_end. At that point it triggers the hard fork config dump and shuts down.

I've added a simple one-node test of this behaviour to the command-line tests, but more should be added in future PRs. The test only validates some of the parameters in the config; it can't quite test the compatible->compatible auto fork behaviour by bringing the daemon up again because I need to refactor the dump_reference_config to allow me to specify that the config should be generated in the Berkeley format. A 2-node nightly integration test based on the existing hard fork test should also be added, but I've deferred that until the go hard fork test is sorted out.

In order to get this test to pass, I had to implement the (previously unimplemented) branches of the auto fork config method that dealt with generating a fork config while the daemon is still at the genesis epoch. I also had to modify the dump method so that it will deal with already-existing config directories a bit better, instead of simply refusing to generate the config if the directory already exists.

Closes: https://github.com/MinaProtocol/mina/issues/18045 Closes: https://github.com/MinaProtocol/mina/issues/18043 Closes: https://github.com/MinaProtocol/mina/issues/17757

[glyh]

I think I've requested to simplify 1439cea8d8497cb8158de956964819b1a3346778 in https://github.com/MinaProtocol/mina/pull/18061

[glyh]

I guess I can come back to this when we've merged https://github.com/MinaProtocol/mina/pull/18061

[glyh]

A 2-node nightly integration test based on the existing hard fork test should also be added, but I've deferred that until the go hard fork test is sorted out.

That sounds exactly like what we should do to hardfork-test-go. Do you know how far are we to merge George's PR?

[glyh]

Is it possible to seperate 1439cea8d8497cb8158de956964819b1a3346778 from the rest of the commits? I think it looks good except I feel back up is too verbose.

Also I'm not confidence how the fork config schema should look like. Do we have some kind of schema, or document on why these fields are needed for forking?

[cjjdespres]

Is it possible to seperate https://github.com/MinaProtocol/mina/commit/1439cea8d8497cb8158de956964819b1a3346778 from the rest of the commits? I think it looks good except I feel back up is too verbose.

Sure, I'll do that.

Also I'm not confidence how the fork config schema should look like. Do we have some kind of schema, or document on why these fields are needed for forking?

No, there's no document that I'm aware of.

The simple answer is that we should only need to set the fields that are present in the current genesis_ledgers/{mainnet,devnet}.json files. Even then, we could probably get away with not setting that many. Theoretically we only need to include the fields that depend directly on the hard fork block in Mesa:

• proof.fork.state_hash
• proof.fork.blockchain_length
• ledger
• epoch_data

But, it's also convenient to set the genesis.genesis_state_timestamp and proof.fork.global_slot_since_genesis in the config; if we relied on compiled defaults for those then we'd have to recompile a mesa binary every time we wanted to schedule a test hard fork.

The daemon.network_id isn't set in the auto fork config because that's still set by the build profile in the compiled config, so the mesa daemon will be able to fill that in on its own.

All the other consensus parameters are compiled defaults that don't need to be included in the output fork config. The ones that are changing can just be updated in advance in the mesa/develop branches.

[cjjdespres]

Hold on, I totally forgot that I put up 1439cea8d8497cb8158de956964819b1a3346778 separately in https://github.com/MinaProtocol/mina/pull/18061/. Sorry about that.

[cjjdespres]

I changed the implementation here so that the config is now saved in auto-fork-config-mesa-{network_name} in the mina config directory. Within that, it saves the fork config to daemon.json and the three genesis ledgers in genesis_ledgers/. I removed the "config regeneration" part of the PR, as it's now unnecessary. The other part (implementing dumping from genesis) was already merged in a separate PR, so I removed that too.

If we do end up wanting in-place config generation (i.e., migrating the existing daemon.json in the mina config directory) it should not be hard to add that.

I will go back and look at the outstanding review comments now.

[glyh]

Looks like except for

I changed the implementation here so that the config is now saved in auto-fork-config-mesa-{network_name} in the mina config directory.

There's nothing new to review? I'm asking because there's a force push rebasing.

[glyh]

I suggest @cjjdespres and @georgeee reach a consensus on the directory hierarchy format though. I don't have a strong opinion here.

[cjjdespres]

There's nothing new to review? I'm asking because there's a force push rebasing.

The only other new thing is that the dump thread uses ~generate_fork_validation:false to generate the fork config (so the fork_validation directory isn't created). I force-pushed to fix some merge conflicts.

[cjjdespres]

I think those should be the rest of the changes, other than improving the test I added.

[glyh]

!ci-build-me

[glyh]

This probably deserves a changelog entry

[cjjdespres]

This probably deserves a changelog entry

Yes. Forgot to push it.

[cjjdespres]

!ci-build-me