starknet-devnet-rs
starknet-devnet-rs copied to clipboard
CI workflow sometimes overuses memory
Sometimes Devnet's CircleCI workflow fails because of memory overuse. Maybe every month or so I see a failure with a message like this:
= note: collect2: fatal error: ld terminated with signal 9 [Killed]
compilation terminated.
I'm pretty sure it's due to parallelization (related Stack Overflow post). Currently we use 7 parallel cargo jobs to test the code: https://github.com/0xSpaceShard/starknet-devnet-rs/blob/37dc6e698297cd5173234397e77a72a756bfa270/.circleci/config.yml#L70
So far, it has always been sufficient to simply restart the failed workflow (example of a successful restart)
This is the memory usage of the passing restarted workflow (peaking at 98% memory used):
Perhaps we could set the number of jobs/cores to 6 instead of 7, but that would probably slow down the testing. Anyway, consider this a heads-up that you might need to restart until something else is implemented.
Caching target/debug/.fingerprint, target/debug/deps, target/debug/incremental didn't help when splitting building and testing into 2 different jobs.
Changing the number of jobs for running integration tests produced the following results:
cargo test --test '*' --jobs 4 -> https://app.circleci.com/pipelines/github/0xSpaceShard/starknet-devnet-rs/1532/workflows/12a6ae28-e5d4-4170-abb0-d186991b1604
cargo test --test '*' --jobs 3 -> https://app.circleci.com/pipelines/github/0xSpaceShard/starknet-devnet-rs/1530/workflows/da5376e5-a9b1-44e6-beee-4037876a116b
Closed via https://github.com/0xSpaceShard/starknet-devnet-rs/pull/554