temper icon indicating copy to clipboard operation
temper copied to clipboard
verified publisher icon EnsoFinance

Setting timeouts

Open banky opened this issue 2 years ago • 3 comments

Is it possible to set a timeout for simulations? If it's not currently implemented, perhaps someone can guide me on how I'd do this? (I'm new to Rust) After running a large number of tx simulations, the requests start to take too long and begin to pile up. It starts out fine, each request takes about 500ms, but then after a few hours, they start to take almost 2 minutes sometimes

 INFO  ts::api > 127.0.0.1:41146 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 39.328261509s
 INFO  ts::api > 127.0.0.1:34204 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 41.263612718s
 INFO  ts::api > 127.0.0.1:35974 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 40.302425564s
 INFO  ts::api > 127.0.0.1:47990 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 39.711832523s
 INFO  ts::api > 127.0.0.1:34482 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 38.599683218s
 INFO  ts::api > 127.0.0.1:45208 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 41.888975767s
 INFO  ts::api > 127.0.0.1:54828 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 44.406452195s
 INFO  ts::api > 127.0.0.1:52346 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 40.537302536s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000000b0587c3c4e3da8_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:54074 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 39.372460717s
 INFO  ts::api > 127.0.0.1:49778 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 48.725948585s
 INFO  ts::api > 127.0.0.1:41092 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 40.484692566s
 INFO  ts::api > 127.0.0.1:54498 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 38.929635772s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000010408fd1300d2a78_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:36648 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 37.135938427s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x000000000000000000000000000000000000000000000000032fd9f90ce7b708_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:34276 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 38.96035323s
 INFO  ts::api > 127.0.0.1:43702 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 50.652176765s
 INFO  ts::api > 127.0.0.1:47860 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 41.573264073s
 INFO  ts::api > 127.0.0.1:56966 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 52.65881698s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000000e943de80380e58_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:60000 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 40.21847157s
 INFO  ts::api > 127.0.0.1:52894 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 38.013594564s
 INFO  ts::api > 127.0.0.1:43714 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 39.502471144s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x000000000000000000000000000000000000000000000000008f388179435478_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:54870 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 37.79750745s
 INFO  ts::api > 127.0.0.1:34278 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 40.057100279s
 INFO  ts::api > 127.0.0.1:41130 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 41.913044435s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x000000000000000000000000000000000000000000000000a0933d25f34875f7_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000001eac882c453d710_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:52922 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 38.517212262s
 INFO  ts::api > 127.0.0.1:50130 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 36.505642283s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000001851cc637109ae5_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:50246 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 37.840279734s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000026562b8d2de144b8_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000082e31d6d93184ba0_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:47888 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 38.80887147s
 INFO  ts::api > 127.0.0.1:34352 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 39.649646456s
 INFO  ts::api > 127.0.0.1:41336 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 38.323364994s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000001702f163c32e9c8_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:34462 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 38.237915451s
 INFO  ts::api > 127.0.0.1:52456 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 39.019137944s
 INFO  ts::api > 127.0.0.1:34290 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 58.059195845s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000012acfb4d67706768_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:40912 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 38.001337322s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000003a0af0d31394040_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:58192 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 37.081051367s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000002f70e7290757df4_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:60226 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 36.945840985s
 INFO  ts::api > 127.0.0.1:54162 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 37.452374699s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000000059180276515598_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:37472 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 34.201172818s
 INFO  ts::api > 127.0.0.1:35338 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 38.607602289s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000001f39791815a32e58_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:34142 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 36.29515275s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000000388a24f1ad5ac40_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:33440 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 37.593369267s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000013ec6f169773e480_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:36688 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 31.780975829s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x000000000000000000000000000000000000000000000000085282127ef409d0_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:58228 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 35.625268199s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x000000000000000000000000000000000000000000000000032fd0f103732c40_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:56932 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 37.554852511s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x000000000000000000000000000000000000000000000000010f2cbcc7659bcc_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:54110 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 33.451618992s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000447c5c4444da7ec40_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000023abb281e2c207a0_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:33270 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 34.061162239s
 INFO  ts::api > 127.0.0.1:34400 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 33.075862895s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000002257262c98c0f58_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:56902 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 32.093076131s
 INFO  ts::api > 127.0.0.1:51792 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 34.969364768s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000000170ccd587a6f0fe_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:52914 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 33.378202003s
 INFO  ts::api > 127.0.0.1:50202 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 46.514351044s
 INFO  ts::api > 127.0.0.1:47944 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 34.343456828s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x00000000000000000000000000000000000000000000000016e9650bc20e4c40_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:34298 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 33.861438118s
[src/evm.rs:240] &err = "backend: failed while inspecting: Transaction(LackOfFundForGasLimit { gas_limit: 0x0000000000000000000000000000000000000000000000000161f6a156a62c40_U256, balance: 0x0000000000000000000000000000000000000000000000000000000000000000_U256 })"
 INFO  ts::api > 127.0.0.1:41150 "POST /api/v1/simulate-bundle HTTP/1.1" 500 "-" "Bun/1.0.11" 31.956652852s
 INFO  ts::api > 127.0.0.1:34322 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 31.983531479s
 INFO  ts::api > 127.0.0.1:57008 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 51.765085951s
 INFO  ts::api > 127.0.0.1:60466 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 102.407593516s
 INFO  ts::api > 127.0.0.1:36792 "POST /api/v1/simulate-bundle HTTP/1.1" 200 "-" "Bun/1.0.11" 128.916109934s

banky avatar Nov 18 '23 14:11 banky

I've forked the repo to try out adding timeouts if anyone else has this issue. I initially thought that the tokio::time::timeout function would correctly check that the function does not take too long. But there is some long running non-async stuff so the timeout doesn't work. I got around this by adding a bunch of tokio::time::sleep(tokio::time::Duration::from_millis(1)).await; to serve as yield points. Any help would be appreciated if someone knows a better way to do this!

https://github.com/banky/temper

banky avatar Nov 20 '23 23:11 banky

Instead of random sleeps, I believe yeld_now is the right thing to use to force a long running task to hand back execution to the runtime. Ideally, you'd find the loops for which each iteration is taking long and add a yield at the end of each round.

fleupold avatar Nov 21 '23 08:11 fleupold

Thank you for the assistance, I'll continue to dig into this. Even with the timeouts, the requests eventually start failing so there is some underlying issue that I'm not seeing

banky avatar Nov 21 '23 13:11 banky