optimize memory consumption for verifier subprocess

[dkijania]

Implementation of https://www.notion.so/o1labs/RAM-savvy-Mina-node-114e79b1f91080968c47e815124a6279 regarding verifier/prover key duplication. Removed duplicated key generation in verifier process and reusing prover verification key (through GRPC) in verifier. Steps are pretty simple:

• Start prover process
• Let prover process generate the proving key and return the verification key back to main thread (via an RPC response)
• Start a verifier process and initiailize Mina node with the verification key returned by the prover process

Testing:

Apart from full suite of integration tests i did tests locally (also in order to generate memory usage comparison before and after changed). For this purpose I used single node (seed without bp keys). Bootstrapped it and observed memory usage for mina and both sub-processes (verifier, prover). Before my changes prover and verifier were using proving keys in a lazy way. Currently prover works more eagerly, that's why mina takes up to 2 GB when bootstrapping, then GC frees memory leaving on only:

• 0.8 GB for prover
• 0.3 GB for main mina process
• 0.2 GB of verifier

To compare previously memory usage distribution was similar to:

• 0.2 GB for prover
• 0.3 GB for main mina process
• 0.2 GB of verifier

Before first usage of prover/verifier cryptographic material

Also what is worth mentioning that due to passing verification key from prover to verifier we gained ~ 15 seconds in bootstrap speed. In order to reach ~1 second we need to refactor :

Pickles.Side_loaded.srs_precomputation () ;

And bake those computation into rust binary. As Matthew mentioned , we can compute it at compile time and completely remove 10s form verifier boatload leaving only 1 sec.

---

Another test was to use local mina network in order to simulate small network method usage. I used network of:

• 1 seed
• 2 whales bp
• 2 fish bp
• 1 passive node
• snark coordinator with 2 snark workers

Below some raw data of memory usage: metrics_new.csv metrics_compatible.csv

And some graphs:

Above graph show that verifiers processes are taking up to 0.9 GB after they are forced to be evaluated. Mina processes are taking around ~250 to 300 Mb of residential memory size. Only two provers (for mina whales) are taking up to 2.0 GB as they are asked to init calculation for produced blocks.

Laser focus on verifiers:

After optimization:

Here we can observe quite big spike of memory for seed (up to 1.7 GB) as it was first node to calculate proof data. Provers are acting quite the same as before change for active block producers. Inactive bp (seeds,fishes,passive nodes are taking up to 0.9 as they eagerly loaded proof data). This behavior could be tweak as well and as a result we could skip launching provers for non-block producers (seeds, archive node feeders) save ~ 1 GB of memory The major change is that more main and verifier processes are taking around 250 MB.

More detailed look at verifiers processes:

And finally verifiers memory comparison: