Berkeley archive migration verifier

[dkijania]

Introduced berkeley_migration_verifier app, which automates checks for archive migration after during and before fork. Please see readme for more details

[ghost-not-in-the-shell]

There're much easier way to make sure all migrated blocks are canonical. And you actually don't need 2 database. For example, you could use this sql to get the number of blocks on the canonical chain from genesis to fork block:

WITH RECURSIVE chain AS 
(  
   SELECT id, parent_id, chain_status FROM blocks 
   WHERE height = <fork-height> AND chain_status = 'canonical' 
   
   UNION ALL 

   SELECT [b.id](http://b.id/), b.parent_id, b.chain_status FROM blocks b
   INNER JOIN chain ON [b.id](http://b.id/) = chain.parent_id AND ([chain.id](http://chain.id/) <> 0 OR [b.id](http://b.id/) = 0)
) SELECT count(*) FROM chain where chain_status = 'canonical';

And compute the result with the total number blocks in the db.

SELECT count(*) FROM blocks;

This way you verified the continuity of the chain and also verified that there's only this canonical chain of blocks in the migrated database.

We should add the check for accounts_accessed and accounts_created table. You can refer to how I did it in the notion doc: https://www.notion.so/o1labs/Hard-fork-runbooks-cb698d4798504ae88c001fd1fe17aea7?p=5ab68e47ab5843db9346fbfa027afc85&pm=s

For the checkpoint, I have a quick and dirty bash script to compare it against the fork config, which I think should be more accurate than the replayer output from mainnet: https://github.com/MinaProtocol/mina/blob/berkeley/scripts/compare-replayer-and-fork-config.sh If you are not satisfied with this, you could directly take the ledger from the checkpoint file and ledger from the new runtime_config (generated from the fork config), then throw those 2 ledgers directly into the Runtime_config module (because they are both generated from this) and do a direct comparison with that. Every field should be equal (up to isomorphism). This would be a better test.

[dkijania]

!ci-build-me

[SanabriaRusso]

Infra part mostly unchanged. Will we be launching this to a Kubernetes cluster? @dkijania

[ghost-not-in-the-shell]

Can you add in the description on how this app being tested?

[ghost-not-in-the-shell]

Another question would be what's the intended use case for this app?

Is it trying to automate the check we do during the migration process?

In the migration runbook: https://www.notion.so/o1labs/Hard-fork-runbooks-cb698d4798504ae88c001fd1fe17aea7?p=5ab68e47ab5843db9346fbfa027afc85&pm=s

There are 2 steps where we valid the migrated database: A10 and A17. The difference is that A10 doesn't assume the existence of a fork config and A17 assumes the validity of A17.

This app contains many checks that's already been covered in A10 but it can only be executed during A17 because it requires a fork config.

Plus the checks for post-fork blocks seems all very superficial, do we really need it? And adding those checks make it even more confusing to me where this app stands in the migration process.

[ghost-not-in-the-shell]

Another question needs to be consider is how much time would this verifier take for a database of mainnet size. Queries for user commands could take 10-20 mins on mainnet.

[ghost-not-in-the-shell]

Currently we have 12 checks, some of these checks are redundant. We are trying to do the following 4 checks on the migrated database:

1. checks the canonical chain on mainnet is the same as the canonical chain on migrated (check 4, 6, 8, 9, 10 should be all covered by this)

This check can be done by a single query like below on both mainnet and migrated database and compare the results.

WITH RECURSIVE chain AS 
(  
   SELECT id, parent_id, chain_status, state_hash, ledger_hash FROM blocks 
   WHERE height = <fork-height> AND chain_status = 'canonical' 
   
   UNION ALL 

   SELECT b.id, b.parent_id, b.chain_status, b.state_hash, b.ledger_hash FROM blocks b
   INNER JOIN chain ON b.id = chain.parent_id AND (chain.id <> 0 OR b.id = 0)
) SELECT state_hash, ledger_hash FROM chain where chain_status = 'canonical';

2. there's no orphaned/pending block in migrated (this should be a simple check that select count(*) from blocks where chain_status = 'orphaned' or chain_status = 'pending' = 0)

3. checks the canonical user command hash on mainnet = user command hash on migrated (already covered by check 2)

4. checks the canonical internal command hash on mainnet = internal command hash on migrated (already covered by check 3)

5. checks accounts accessed (covered by check 5&7, and it needs to be updated)

note that 3&4 assumes that the chain that leads to the fork block is canonical on mainnet. This assumption is not true on all mainnet archive dump.

A side note: since all the checks are performing using just sql and caqti seems to be slow when retrieving results from sql queries. We could consider using another language to achieve this task.

[dkijania]

!ci-build-me

[dkijania]

!ci-build-me

[dkijania]

!ci-build-me

[dkijania]

!ci-build-me

[emberian]

Can we add this to any of our cron jobs?

Or some other kind of CI? I don't want to see this drift.

[dkijania]

verifier will be packed with other migration tooling once this is merged:

https://github.com/MinaProtocol/mina/pull/15248

[dkijania]

!ci-build-me

[mrmr1993]

!approved-for-mainnet