succinctlabs
Increase memory limit
Component
sp1-zkvm
Describe the feature you would like
SP1 has memory limit of 1-2GB (depending on the allocator). Grandine is snarkifying Beacon Chain and this limit is not enough for mainnet size STF.
Additional context
No response
It seems like the bump allocator is the one with the memory limits — see here. Note that the panic!("Memory limit exceeded (0x78000000)") only occurs when the embedded feature is not enabled.
From the docs, it looks like the embedded allocator is intended to overcome those limits: https://docs.succinct.xyz/docs/sp1/generating-proofs/advanced#embedded-allocator
Hi everyone, thanks for highlighting this issue. I’d like to add that enabling the embedded-allocator by default or at least warning users when it's not enabled and memory is about to exceed limits might improve developer experience significantly.
It might also help to:
Detect the allocator type at runtime or build time and provide a clear error message when the embedded allocator is missing.
Offer a CLI flag or config option that explicitly toggles embedded-allocator, with documentation pointing to its purpose and limitations.
Let me know if this is something worth exploring in a PR I’d be happy to help contribute.
For Grandine STF, even embedded allocator is not enough to prove mainnet state. Enabling it via "embedded" feature like this:
sp1-zkvm = { version = '5.2.1', features = ['embedded'] }
At some point, error occurs:
stderr: memory allocation of 239075332 bytes failed
Error: got unimplemented as opcode
