George Agapov

> Mina command to export ledger(s) (without need to start a node) It's possible to export genesis ledger, as well as epoch ledgers from genesis config. Export of staged ledger...

Findings to the date: Some nodes shutdown and take a very long time to recover. Reasons for shutdown are: 1. OOM: in this case there is an abrupt end of...

One log that might be related: ``` 2024-01-19 22:10:05.842323Z] Long async cyc 643.546315777 ["serve_client_rpcs","mina"] [2024-01-19 22:10:05.844397Z] Long async job 5.247238187 ["dispatch_user_command_verification","build_breadcrumb","handle_super_catchup_jobs","perform_super_catchup","transition_router","mina_lib","mina"] [2024-01-19 22:15:56.092503Z] Long async cyc 226.315262121 ["manage_verifier_subprocess","mina_lib","mina"] [2024-01-19 22:15:56.115380Z]...

Another instance, in logs provided by user ak 47: ``` [2024-01-21 14:41:35.596298Z] Long async cyc 630.380755318 ["node_status_service"] [2024-01-21 14:41:35.598062Z] Long async job 630.347027474 [] [2024-01-21 14:58:44.374674Z] Long async cyc 667.632535441...

From niuniu: ``` [2024-01-21 10:25:49.236056Z] Long async cyc 15.101878393 [] [2024-01-21 10:25:49.236086Z] Long async job 15.101868334 [] ```

Seems like nearly all of the logs analyzed contain instances of `Gc.compact` taking `10-30s` or more, in one occasion all the way up to 30 minutes. At least in some...

Logs: [dir1](https://drive.google.com/drive/folders/1rueL6RxRFKsZRmiXfVR2C27B3sMSEVXI2g13EII-S_HGyxxp36vW35YvO_5f4BxIIJP02h0_), [dir2](https://drive.google.com/drive/folders/1xwkqj1EX7-Yq3meFPz2NebvnpMHAwlH54xSE1PCPft-jxvazw7LHED-EJHLvt3l_u7Rmqt4L?usp=sharing)

Tests described in this task were implemented as part of HF test that lives in nightly CI.

An alternative path was taken and this issue is no longer relevant

!ci-build-me