Fuzzing tests occasionally panic with "arithmetic operation overflow" and "out of range"
Component
Forge
Have you ensured that all of these are up to date?
- [X] Foundry
- [X] Foundryup
What version of Foundry are you on?
forge 0.2.0 (1d9a34e 2023-03-07T00:07:40.652148Z)
What command(s) is the bug in?
forge test
Operating System
macOS (Apple Silicon)
Describe the bug
Running into occasional panics when running forge test -- sometimes tests run just fine, sometimes they panic with an arithmetic operation overflow
Initially I thought this was from running the latest nightly, but I'm seeing it happen with commits as far back as 0453ac898150133f5fc2eecf0bf7871cd392d0b1
Here's the panic when using the nightly binary -- the backtrace isn't very helpful, see below for a more helpful backtrace:
$ RUST_BACKTRACE=full FOUNDRY_DEBUG=true yarn test:forge
[⠒] Compiling...
No files changed, compilation skipped
Running 3 tests for test/PausableReentrancyGuard.t.sol:PausableReentrancyGuardTest
[PASS] testNonreentrant() (gas: 9628)
The application panicked (crashed).
Message: arithmetic operation overflow
Location: /Users/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/primitive-types-0.12.1/src/lib.rs:38
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
1: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
2: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
3: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
4: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
5: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
6: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
7: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
8: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>[PASS] testNonreentrantNotPaused() (gas: 14163)
9: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
10: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
11: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
12: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
13: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
14: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
15: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
16: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
17: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
18: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
19: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
20: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
21: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
22: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
23: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
24: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
25: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
26: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
27: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
28: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
29: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
30: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
31: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
32: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
33: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
34: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
35: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
36: __mh_execute_header<unknown>
at <unknown source file>:<unknown line>
37: __pthread_deallocate<unknown>
at <unknown source file>:<unknown line>
Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
And when building from source (backtrace is a bit prettier):
$ RUST_BACKTRACE=full FOUNDRY_DEBUG=true yarn test:forge
[⠒] Compiling...
No files changed, compilation skipped
Running 3 tests for test/PausableReentrancyGuard.t.sol:PausableReentrancyGuardTest
[PASS] testNonreentrant() (gas: 9628)
[PASS] testNonreentrantNotPaused() (gas: 14163)
[PASS] testPause() (gas: 13635)
Test result: ok. 3 passed; 0 failed; finished in 646.67µs
Running 7 tests for test/igps/gas-oracles/StorageGasOracle.t.sol:StorageGasOracleTest
[PASS] testConstructorSetsOwnership() (gas: 7611)
[PASS] testGetExchangeRateAndGasPrice() (gas: 12456)
[PASS] testGetExchangeRateAndGasPriceUnknownDomain() (gas: 8064)
[PASS] testSetRemoteGasData() (gas: 38836)
[PASS] testSetRemoteGasDataConfigs() (gas: 69238)
[PASS] testSetRemoteGasDataConfigsRevertsIfNotOwner() (gas: 12227)
[PASS] testSetRemoteGasDataRevertsIfNotOwner() (gas: 11275)
The application panicked (crashed).
Message: arithmetic operation overflow
Location: /Users/trevor/.cargo/registry/src/github.com-1ecc6299db9ec823/primitive-types-0.12.1/src/lib.rs:38
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⋮ 9 frames hidden ⋮
10: <primitive_types::U256 as core::ops::arith::SubAssign>::sub_assign::haaab349295433f7d
at <unknown source file>:<unknown line>
11: revm::journaled_state::JournaledState::journal_revert::hbfb24b5629c429f7
at <unknown source file>:<unknown line>
12: revm::journaled_state::JournaledState::checkpoint_revert::hf724256058e15241
at <unknown source file>:<unknown line>
13: revm::evm_impl::EVMImpl<GSPEC,DB,_>::call_inner::h20764f8d08684cf2
at <unknown source file>:<unknown line>
14: revm::instructions::host::call::hf74a6a3954a3c87c
at <unknown source file>:<unknown line>
15: revm::interpreter::Interpreter::run::h9a1b13fb755d2572
at <unknown source file>:<unknown line>
16: revm::evm_impl::EVMImpl<GSPEC,DB,_>::call_inner::h20764f8d08684cf2
at <unknown source file>:<unknown line>
17: <revm::evm_impl::EVMImpl<GSPEC,DB,_> as revm::evm_impl::Transact>::transact::hfad8a2d76e9fa1b8
at <unknown source file>:<unknown line>
18: foundry_evm::executor::Executor::call_raw::h2b69b604da620b17
at <unknown source file>:<unknown line>
19: core::ops::function::impls::<impl core::ops::function::Fn<A> for &F>::call::h511e6c79382ced78
at <unknown source file>:<unknown line>
20: proptest::test_runner::runner::call_test::h81cb0f3b7b66bbc7
at <unknown source file>:<unknown line>
21: proptest::test_runner::runner::TestRunner::gen_and_run_case::hb51e7db8c19b0f79
at <unknown source file>:<unknown line>
22: proptest::test_runner::runner::TestRunner::run_in_process::h8c670db8ce325f4a
at <unknown source file>:<unknown line>
23: proptest::test_runner::runner::TestRunner::run::h99241cdac7080f3d
at <unknown source file>:<unknown line>
24: foundry_evm::fuzz::FuzzedExecutor::fuzz::h2e2cc71dbb9d80f6
at <unknown source file>:<unknown line>
25: forge::runner::ContractRunner::run_fuzz_test::h2114219aaf246d4a
at <unknown source file>:<unknown line>
26: rayon::iter::plumbing::Folder::consume_iter::h75bade315edc67d6
at <unknown source file>:<unknown line>
27: rayon::iter::plumbing::bridge_producer_consumer::helper::hbea1493a0aa54248
at <unknown source file>:<unknown line>
28: rayon_core::registry::in_worker::hc879f2ba68ac5ddb
at <unknown source file>:<unknown line>
29: rayon::iter::plumbing::bridge_producer_consumer::helper::hbea1493a0aa54248
at <unknown source file>:<unknown line>
30: <rayon::iter::flat_map::FlatMap<I,F> as rayon::iter::ParallelIterator>::drive_unindexed::hc2453796c8fcf645
at <unknown source file>:<unknown line>
31: rayon::result::<impl rayon::iter::FromParallelIterator<core::result::Result<T,E>> for core::result::Result<C,E>>::from_par_iter::h10c67b0f765d8ca5
at <unknown source file>:<unknown line>
32: forge::runner::ContractRunner::run_tests::hafa155cf36f03529
at <unknown source file>:<unknown line>
33: forge::multi_runner::MultiContractRunner::run_tests::h96e6ebd818814301
at <unknown source file>:<unknown line>
34: <rayon::iter::map::MapFolder<C,F> as rayon::iter::plumbing::Folder<T>>::consume::hc897c0c123a40d4e
at <unknown source file>:<unknown line>
35: rayon::iter::plumbing::Folder::consume_iter::h92773b96e146ddb4
at <unknown source file>:<unknown line>
36: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
37: rayon_core::registry::in_worker::h776a947bb090cf9b
at <unknown source file>:<unknown line>
38: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
39: rayon_core::registry::in_worker::h776a947bb090cf9b
at <unknown source file>:<unknown line>
40: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hc93f2e33d11b7357
at <unknown source file>:<unknown line>
42: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::hba510294234ed2bb
at <unknown source file>:<unknown line>
43: rayon_core::registry::WorkerThread::wait_until_cold::h4d4f4d271f2eaa71
at <unknown source file>:<unknown line>
44: rayon_core::registry::ThreadBuilder::run::h82cf17e75c093122
at <unknown source file>:<unknown line>
45: std::sys_common::backtrace::__rust_begin_short_backtrace::h7a5df0c254a4feee
at <unknown source file>:<unknown line>
46: core::ops::function::FnOnce::call_once{{vtable.shim}}::h217eb88d52405018
at <unknown source file>:<unknown line>
47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hf101e7e0479e2c15
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/boxed.rs:2000
48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hcc97a7a0d1eb566a
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/boxed.rs:2000
49: std::sys::unix::thread::Thread::new::thread_start::h92ee0ad602ca1aab
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys/unix/thread.rs:108
50: __pthread_deallocate<unknown>
at <unknown source file>:<unknown line>
Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Test result: ok. 7 passed; 0 failed; finished in 655.50µs
Running 7 tests for test/igps/OverheadIgp.t.sol:OverheadIgpTest
[PASS] testDestinationGasAmount() (gas: 33814)
The application panicked (crashed).
Message: arithmetic operation overflow
Location: /Users/trevor/.cargo/registry/src/github.com-1ecc6299db9ec823/primitive-types-0.12.1/src/lib.rs:38
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⋮ 9 frames hidden ⋮
10: <primitive_types::U256 as core::ops::arith::SubAssign>::sub_assign::haaab349295433f7d
at <unknown source file>:<unknown line>
11: revm::journaled_state::JournaledState::journal_revert::hbfb24b5629c429f7
at <unknown source file>:<unknown line>
12: revm::journaled_state::JournaledState::checkpoint_revert::hf724256058e15241
at <unknown source file>:<unknown line>
13: revm::evm_impl::EVMImpl<GSPEC,DB,_>::call_inner::h20764f8d08684cf2
at <unknown source file>:<unknown line>
14: revm::instructions::host::call::hf74a6a3954a3c87c
at <unknown source file>:<unknown line>
15: revm::interpreter::Interpreter::run::h9a1b13fb755d2572
at <unknown source file>:<unknown line>
16: revm::evm_impl::EVMImpl<GSPEC,DB,_>::call_inner::h20764f8d08684cf2
at <unknown source file>:<unknown line>
17: <revm::evm_impl::EVMImpl<GSPEC,DB,_> as revm::evm_impl::Transact>::transact::hfad8a2d76e9fa1b8
at <unknown source file>:<unknown line>
18: foundry_evm::executor::Executor::call_raw::h2b69b604da620b17
at <unknown source file>:<unknown line>
19: core::ops::function::impls::<impl core::ops::function::Fn<A> for &F>::call::h511e6c79382ced78
at <unknown source file>:<unknown line>
20: proptest::test_runner::runner::call_test::h81cb0f3b7b66bbc7
at <unknown source file>:<unknown line>
21: proptest::test_runner::runner::TestRunner::gen_and_run_case::hb51e7db8c19b0f79
at <unknown source file>:<unknown line>
22: proptest::test_runner::runner::TestRunner::run_in_process::h8c670db8ce325f4a
at <unknown source file>:<unknown line>
23: proptest::test_runner::runner::TestRunner::run::h99241cdac7080f3d
at <unknown source file>:<unknown line>
24: foundry_evm::fuzz::FuzzedExecutor::fuzz::h2e2cc71dbb9d80f6
at <unknown source file>:<unknown line>
25: forge::runner::ContractRunner::run_fuzz_test::h2114219aaf246d4a
at <unknown source file>:<unknown line>[PASS] testDestinationGasAmountWhenOverheadNotSet() (gas: 7912)
26: rayon::iter::plumbing::Folder::consume_iter::h75bade315edc67d6
at <unknown source file>:<unknown line>
27: rayon::iter::plumbing::bridge_producer_consumer::helper::hbea1493a0aa54248
at <unknown source file>:<unknown line>
28: rayon_core::registry::in_worker::hc879f2ba68ac5ddb
at <unknown source file>:<unknown line>
29: rayon::iter::plumbing::bridge_producer_consumer::helper::hbea1493a0aa54248
at <unknown source file>:<unknown line>
30: <rayon::iter::flat_map::FlatMap<I,F> as rayon::iter::ParallelIterator>::drive_unindexed::hc2453796c8fcf645
at <unknown source file>:<unknown line>
31: rayon::result::<impl rayon::iter::FromParallelIterator<core::result::Result<T,E>> for core::result::Result<C,E>>::from_par_iter::h10c67b0f765d8ca5
at <unknown source file>:<unknown line>
32: forge::runner::ContractRunner::run_tests::hafa155cf36f03529
at <unknown source file>:<unknown line>
33: forge::multi_runner::MultiContractRunner::run_tests::h96e6ebd818814301
at <unknown source file>:<unknown line>
34: <rayon::iter::map::MapFolder<C,F> as rayon::iter::plumbing::Folder<T>>::consume::hc897c0c123a40d4e
at <unknown source file>:<unknown line>
35: rayon::iter::plumbing::Folder::consume_iter::h92773b96e146ddb4
at <unknown source file>:<unknown line>
36: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
37: rayon_core::registry::in_worker::h776a947bb090cf9b
at <unknown source file>:<unknown line>
38: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
39: rayon_core::registry::in_worker::h776a947bb090cf9b
at <unknown source file>:<unknown line>
40: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
41: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hc93f2e33d11b7357
at <unknown source file>:<unknown line>
42: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::hba510294234ed2bb
at <unknown source file>:<unknown line>
43: rayon_core::registry::WorkerThread::wait_until_cold::h4d4f4d271f2eaa71
at <unknown source file>:<unknown line>
44: rayon_core::registry::ThreadBuilder::run::h82cf17e75c093122
at <unknown source file>:<unknown line>
45: std::sys_common::backtrace::__rust_begin_short_backtrace::h7a5df0c254a4feee
at <unknown source file>:<unknown line>
46: core::ops::function::FnOnce::call_once{{vtable.shim}}::h217eb88d52405018
at <unknown source file>:<unknown line>
47: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hf101e7e0479e2c15
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/boxed.rs:2000
48: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hcc97a7a0d1eb566a
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/boxed.rs:2000
49: std::sys::unix::thread::Thread::new::thread_start::h92ee0ad602ca1aab
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys/unix/thread.rs:108
50: __pthread_deallocate<unknown>
at <unknown source file>:<unknown line>
Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
^ this continues on for quite some time with a bunch more panics with the same message, however I'm now seeing the final panic message is different:
The application panicked (crashed).
Message: range start index 4 out of range for slice of length 0
Location: evm/src/fuzz/mod.rs:200
━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ BACKTRACE ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━
⋮ 9 frames hidden ⋮
10: core::slice::index::slice_start_index_len_fail_rt::hc4b7c5ac5f48485d
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/slice/index.rs:53
11: core::slice::index::slice_start_index_len_fail::he934197ee6ba25e2
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/core/src/slice/index.rs:41
12: foundry_evm::fuzz::FuzzedExecutor::fuzz::h2e2cc71dbb9d80f6
at <unknown source file>:<unknown line>
13: forge::runner::ContractRunner::run_fuzz_test::h2114219aaf246d4a
at <unknown source file>:<unknown line>
14: rayon::iter::plumbing::Folder::consume_iter::h75bade315edc67d6
at <unknown source file>:<unknown line>
15: rayon::iter::plumbing::bridge_producer_consumer::helper::hbea1493a0aa54248
at <unknown source file>:<unknown line>
16: <rayon::iter::flat_map::FlatMap<I,F> as rayon::iter::ParallelIterator>::drive_unindexed::hc2453796c8fcf645
at <unknown source file>:<unknown line>
17: rayon::result::<impl rayon::iter::FromParallelIterator<core::result::Result<T,E>> for core::result::Result<C,E>>::from_par_iter::h10c67b0f765d8ca5
at <unknown source file>:<unknown line>
18: forge::runner::ContractRunner::run_tests::hafa155cf36f03529
at <unknown source file>:<unknown line>
19: forge::multi_runner::MultiContractRunner::run_tests::h96e6ebd818814301
at <unknown source file>:<unknown line>
20: <rayon::iter::map::MapFolder<C,F> as rayon::iter::plumbing::Folder<T>>::consume::hc897c0c123a40d4e
at <unknown source file>:<unknown line>
21: rayon::iter::plumbing::Folder::consume_iter::h92773b96e146ddb4
at <unknown source file>:<unknown line>
22: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
23: rayon_core::registry::in_worker::h776a947bb090cf9b
at <unknown source file>:<unknown line>
24: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
25: rayon_core::registry::in_worker::h776a947bb090cf9b
at <unknown source file>:<unknown line>
26: rayon::iter::plumbing::bridge_producer_consumer::helper::h9c37dde9b9d23459
at <unknown source file>:<unknown line>
27: <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hc93f2e33d11b7357
at <unknown source file>:<unknown line>
28: <rayon_core::job::StackJob<L,F,R> as rayon_core::job::Job>::execute::hba510294234ed2bb
at <unknown source file>:<unknown line>
29: rayon_core::registry::WorkerThread::wait_until_cold::h4d4f4d271f2eaa71
at <unknown source file>:<unknown line>
30: rayon_core::registry::ThreadBuilder::run::h82cf17e75c093122
at <unknown source file>:<unknown line>
31: std::sys_common::backtrace::__rust_begin_short_backtrace::h7a5df0c254a4feee
at <unknown source file>:<unknown line>
32: core::ops::function::FnOnce::call_once{{vtable.shim}}::h217eb88d52405018
at <unknown source file>:<unknown line>
33: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hf101e7e0479e2c15
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/boxed.rs:2000
34: <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hcc97a7a0d1eb566a
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/alloc/src/boxed.rs:2000
35: std::sys::unix::thread::Thread::new::thread_start::h92ee0ad602ca1aab
at /rustc/d5a82bbd26e1ad8b7401f6a718a9c57c96905483/library/std/src/sys/unix/thread.rs:108
36: __pthread_deallocate<unknown>
at <unknown source file>:<unknown line>
Run with COLORBT_SHOW_HIDDEN=1 environment variable to disable frame filtering.
Seems related to fuzzing. Going to continue debugging & playing around with our fuzzing tests specifically to see if I can nail it down to a specific test case
cc @joshieDo
@tkporter can you pls retry and report back if still an issue (it it is would be great to have instructions to reproduce it locally). thank you
Marking as requires reproduction
@tkporter feel free to re-open with instructions to reproduce or a minimal example if you are still running into this