chirpstack icon indicating copy to clipboard operation
chirpstack copied to clipboard
verified publisher icon chirpstack

Added support for redis sentinel

Open ederuiter opened this issue 1 year ago • 9 comments

Support for redis sentinal was introduced in deadpool-redis 0.17

ederuiter avatar Nov 05 '24 21:11 ederuiter

Unfortunately I was not able to test this locally as the test keep failing with a stack overflow error

I have no clue as to what causes this since the new code should not even be invoked, as I have not added any test case for this.

For me it fails reliably with

thread 'test::class_a_pr_test::test_sns_uplink' has overflowed its stack
fatal runtime error: stack overflow

Stack trace show a deep callstack; but nothing obvious that raises alarms for me:

#0  0x00007f199de55efc in __pthread_kill_implementation () from /nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
#1  0x00007f199de05e86 in raise () from /nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
#2  0x00007f199ddee935 in abort () from /nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
#3  0x000055635b9c6bea in std::sys::pal::unix::abort_internal () at library/std/src/sys/pal/unix/mod.rs:370
#4  0x000055635b9c51b5 in std::sys::pal::unix::stack_overflow::imp::signal_handler () at library/std/src/rt.rs:43
#5  <signal handler called>
#6  0x000055635a22a3e1 in chirpstack::downlink::data::{impl#0}::set_mac_commands::{async_fn#0} () at chirpstack/src/downlink/data.rs:629
#7  0x000055635a21a3f5 in chirpstack::downlink::data::{impl#0}::_handle_response::{async_fn#0} () at chirpstack/src/downlink/data.rs:212
#8  0x00005563596dda92 in tracing::instrument::{impl#2}::poll<chirpstack::downlink::data::{impl#0}::_handle_response::{async_fn_env#0}> (self=..., cx=0x7f199dda1a78) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/instrument.rs:321
#9  0x000055635a215c12 in chirpstack::downlink::data::{impl#0}::handle_response::{async_fn#0} () at chirpstack/src/downlink/data.rs:84
#10 0x000055635a36782a in chirpstack::uplink::data::{impl#0}::start_downlink_data_flow::{async_fn#0} () at chirpstack/src/uplink/data.rs:1301
#11 0x000055635a3420ab in chirpstack::uplink::data::{impl#0}::_handle::{async_fn#0} () at chirpstack/src/uplink/data.rs:157
#12 0x00005563596e05f2 in tracing::instrument::{impl#2}::poll<chirpstack::uplink::data::{impl#0}::_handle::{async_fn_env#0}> (self=..., cx=0x7f199dda1a78) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/instrument.rs:321
#13 0x000055635a02bbe9 in chirpstack::uplink::data_sns::{impl#0}::handle::{async_fn#0} () at chirpstack/src/uplink/data_sns.rs:11
#14 0x0000556359f72bad in chirpstack::api::backend::_handle_pr_start_req_data::{async_fn#0} () at chirpstack/src/api/backend/mod.rs:353
#15 0x0000556359f6fcbb in chirpstack::api::backend::_handle_pr_start_req::{async_fn#0} () at chirpstack/src/api/backend/mod.rs:258
#16 0x0000556359f6d30e in chirpstack::api::backend::handle_pr_start_req::{async_fn#0} () at chirpstack/src/api/backend/mod.rs:237
#17 0x0000556359f6b53b in chirpstack::api::backend::_handle_request::{async_fn#0} () at chirpstack/src/api/backend/mod.rs:173
#18 0x00005563596de63a in tracing::instrument::{impl#2}::poll<chirpstack::api::backend::_handle_request::{async_fn_env#0}> (self=..., cx=0x7f199dda1a78) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tracing-0.1.40/src/instrument.rs:321
#19 0x0000556359f648e4 in chirpstack::api::backend::handle_request::{async_fn#0} () at chirpstack/src/api/backend/mod.rs:97
#20 0x000055635a0b1e04 in chirpstack::test::class_a_pr_test::test_sns_uplink::{async_block#0} () at chirpstack/src/test/class_a_pr_test.rs:388
#21 0x0000556359adcaf3 in core::future::future::{impl#1}::poll<&mut dyn core::future::future::Future<Output=()>> (self=..., cx=0x7f199dda1a78) at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/future/future.rs:123
#22 0x0000556359adf496 in core::future::future::{impl#1}::poll<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> (self=..., cx=0x7f199dda1a78) at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/future/future.rs:123
#23 0x00005563598c8b24 in tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}::{closure#0}::{closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> ()
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:729
#24 0x00005563598c815a in tokio::runtime::coop::with_budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> (budget=..., 
    f=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/coop.rs:107
#25 tokio::runtime::coop::budget<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> (f=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/coop.rs:73
#26 tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}::{closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> ()
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:729
#27 0x00005563598baafc in tokio::runtime::scheduler::current_thread::Context::enter<core::task::poll::Poll<()>, tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>> (
    self=0x7f199dda1fe8, core=0x7f1988002810, f=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:428
#28 0x00005563598c4f4e in tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> (core=0x7f1988002810, context=0x7f199dda1fe8)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:728
#29 0x00005563598c079b in tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>> ()
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:807
#30 0x0000556359dead3b in tokio::runtime::context::scoped::Scoped<tokio::runtime::scheduler::Context>::set<tokio::runtime::scheduler::Context, tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> (self=0x7f199ddbd280, t=0x7f199dda1fe0, f=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/context/scoped.rs:40
#31 0x000055635a1eaa79 in tokio::runtime::context::set_scheduler::{closure#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>> (c=0x7f199ddbd258) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/context.rs:180
#32 0x00005563595da1f6 in std::thread::local::LocalKey<tokio::runtime::context::Context>::try_with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> (self=0x55635c3448a0, f=...) at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/thread/local.rs:286
#33 0x00005563595d7c7a in std::thread::local::LocalKey<tokio::runtime::context::Context>::with<tokio::runtime::context::Context, tokio::runtime::context::set_scheduler::{closure_env#0}<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>>, (alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>)> (self=0x55635c3448a0, f=...) at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/std/src/thread/local.rs:262
#34 0x000055635a1ea67d in tokio::runtime::context::set_scheduler<(alloc::boxed::Box<tokio::runtime::scheduler::current_thread::Core, alloc::alloc::Global>, core::option::Option<()>), tokio::runtime::scheduler::current_thread::{impl#8}::enter::{closure_env#0}<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>>> (v=0x7f199dda1fe0, f=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/context.rs:180
#35 0x00005563598be9a0 in tokio::runtime::scheduler::current_thread::CoreGuard::enter<tokio::runtime::scheduler::current_thread::{impl#8}::block_on::{closure_env#0}<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>>, core::option::Option<()>> (self=..., f=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:807
#36 0x00005563598c0c4d in tokio::runtime::scheduler::current_thread::CoreGuard::block_on<core::pin::Pin<&mut core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>> (self=..., future=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:716
#37 0x00005563598ad54a in tokio::runtime::scheduler::current_thread::{impl#0}::block_on::{closure#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> (blocking=0x7f199dda2180)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:196
#38 0x000055635a4234ea in tokio::runtime::context::runtime::enter_runtime<tokio::runtime::scheduler::current_thread::{impl#0}::block_on::{closure_env#0}<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>>, ()> (handle=0x7f199ddbc130, allow_block_in_place=false, f=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/context/runtime.rs:65
#39 0x00005563598aba6e in tokio::runtime::scheduler::current_thread::CurrentThread::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> (self=0x7f199ddbc108, handle=0x7f199ddbc130, future=...)
    at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/scheduler/current_thread/mod.rs:184
#40 0x00005563595d4d50 in tokio::runtime::runtime::Runtime::block_on_inner<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> (self=0x7f199ddbc100, future=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/runtime.rs:368
#41 0x00005563595d5d8f in tokio::runtime::runtime::Runtime::block_on<core::pin::Pin<&mut dyn core::future::future::Future<Output=()>>> (self=0x7f199ddbc100, future=...) at /root/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.41.0/src/runtime/runtime.rs:342
#42 0x000055635a075c2a in chirpstack::test::class_a_pr_test::test_sns_uplink () at chirpstack/src/test/class_a_pr_test.rs:423
#43 0x000055635a0ac817 in chirpstack::test::class_a_pr_test::test_sns_uplink::{closure#0} () at chirpstack/src/test/class_a_pr_test.rs:174
#44 0x00005563598d7356 in core::ops::function::FnOnce::call_once<chirpstack::test::class_a_pr_test::test_sns_uplink::{closure_env#0}, ()> () at /rustc/129f3b9964af4d4a709d1383930ade12dfe7c081/library/core/src/ops/function.rs:250
#45 0x000055635a56564b in core::ops::function::FnOnce::call_once<fn() -> core::result::Result<(), alloc::string::String>, ()> () at library/core/src/ops/function.rs:250
#46 test::__rust_begin_short_backtrace<core::result::Result<(), alloc::string::String>, fn() -> core::result::Result<(), alloc::string::String>> () at library/test/src/lib.rs:623
#47 0x000055635a564d51 in test::run_test_in_process::{closure#0} () at library/test/src/lib.rs:646
#48 core::panic::unwind_safe::{impl#23}::call_once<core::result::Result<(), alloc::string::String>, test::run_test_in_process::{closure_env#0}> () at library/core/src/panic/unwind_safe.rs:272
#49 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure_env#0}>, core::result::Result<(), alloc::string::String>> () at library/std/src/panicking.rs:559
#50 std::panicking::try<core::result::Result<(), alloc::string::String>, core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure_env#0}>> () at library/std/src/panicking.rs:523
#51 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<test::run_test_in_process::{closure_env#0}>, core::result::Result<(), alloc::string::String>> () at library/std/src/panic.rs:149
#52 test::run_test_in_process () at library/test/src/lib.rs:646
#53 test::run_test::{closure#0} () at library/test/src/lib.rs:569
#54 0x000055635a52d7d4 in test::run_test::{closure#1} () at library/test/src/lib.rs:597
#55 std::sys_common::backtrace::__rust_begin_short_backtrace<test::run_test::{closure_env#1}, ()> () at library/std/src/sys_common/backtrace.rs:155
#56 0x000055635a532242 in std::thread::{impl#0}::spawn_unchecked_::{closure#2}::{closure#0}<test::run_test::{closure_env#1}, ()> () at library/std/src/thread/mod.rs:542
#57 core::panic::unwind_safe::{impl#23}::call_once<(), std::thread::{impl#0}::spawn_unchecked_::{closure#2}::{closure_env#0}<test::run_test::{closure_env#1}, ()>> () at library/core/src/panic/unwind_safe.rs:272
#58 std::panicking::try::do_call<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#2}::{closure_env#0}<test::run_test::{closure_env#1}, ()>>, ()> () at library/std/src/panicking.rs:559
#59 std::panicking::try<(), core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#2}::{closure_env#0}<test::run_test::{closure_env#1}, ()>>> () at library/std/src/panicking.rs:523
#60 std::panic::catch_unwind<core::panic::unwind_safe::AssertUnwindSafe<std::thread::{impl#0}::spawn_unchecked_::{closure#2}::{closure_env#0}<test::run_test::{closure_env#1}, ()>>, ()> () at library/std/src/panic.rs:149
#61 std::thread::{impl#0}::spawn_unchecked_::{closure#2}<test::run_test::{closure_env#1}, ()> () at library/std/src/thread/mod.rs:541
#62 core::ops::function::FnOnce::call_once<std::thread::{impl#0}::spawn_unchecked_::{closure_env#2}<test::run_test::{closure_env#1}, ()>, ()> () at library/core/src/ops/function.rs:250
#63 0x000055635b9c58eb in alloc::boxed::{impl#48}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2022
#64 alloc::boxed::{impl#48}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at library/alloc/src/boxed.rs:2022
#65 std::sys::pal::unix::thread::{impl#2}::new::thread_start () at library/std/src/sys/pal/unix/thread.rs:108
#66 0x00007f199de54272 in start_thread () from /nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6
#67 0x00007f199decfc24 in clone () from /nix/store/c10zhkbp6jmyh0xc5kd123ga8yy2p4hk-glibc-2.39-52/lib/libc.so.6

@brocaar any idea what might cause this? My only suspicion is that this adds an extra dependency for deadpool-redis (tokio) that causes some behavioral change; but according to the docs of deadpool it should not

ederuiter avatar Nov 05 '24 21:11 ederuiter

Thanks @ederuiter :+: Lets see if running the test using GitHub Actions raises the same stack overflow.

brocaar avatar Nov 06 '24 11:11 brocaar

failed :-( the good thing it seems to be reproducible :-|

Another issue is that currently the database selection fails if you use another database number than 0 as it tries to select the database on the sentinel; not on the actual redis connection I think it should support it when configuring the pool; but documentation is not the best and I have no experience with rust/redis-rs/deadpool .. do you have any idea how to achieve this?

ederuiter avatar Nov 06 '24 13:11 ederuiter

@brocaar It seems that for some reason my changes increased the used stack space slightly .. just to push it over the limit for this test

Debugging with gdb seems to show that the used stack space up until the overflow (see https://stackoverflow.com/a/37327019) is approximately 2MB:

make test
...
gdb --args /home/eric/code/oss/chirpstack/target/debug/deps/chirpstack-f5322796d11f3a3f test::class_a_pr_test::test_sns_uplink
...
(gdb) r
Thread 2 "test::class_a_p" received signal SIGSEGV, Segmentation fault.
[Switching to LWP 3333828]
0x00005555567f73e1 in chirpstack::downlink::data::Data::set_mac_commands::{{closure}} ()
    at chirpstack/src/downlink/data.rs:629
629	    async fn set_mac_commands(&mut self) -> Result<()> {
(gdb) frame 0
#0  0x00005555567f73e1 in chirpstack::downlink::data::Data::set_mac_commands::{{closure}} ()
    at chirpstack/src/downlink/data.rs:629
629	    async fn set_mac_commands(&mut self) -> Result<()> {
(gdb) set $topsp=$sp
(gdb) bt
...
#60 0x00007ffff7d59272 in start_thread () from /nix/store/kpy2cyd05vdr6j1h200av81fnlxl1jw0-glibc-2.39-52/lib/libc.so.6
#61 0x00007ffff7dd4dec in clone3 () from /nix/store/kpy2cyd05vdr6j1h200av81fnlxl1jw0-glibc-2.39-52/lib/libc.so.6
(gdb) frame 61
#61 0x00007ffff7dd4dec in clone3 () from /nix/store/kpy2cyd05vdr6j1h200av81fnlxl1jw0-glibc-2.39-52/lib/libc.so.6
(gdb) print $sp - $topsp
$1 = 2094680

running the tests with RUST_MIN_STACK=2621440 (2.5MB) seem to work without issue

This could be caused by the many .await statements in this test as

Async Rust keeps everything on stack across await points

(according to: https://www.reddit.com/r/rust/comments/10l99ai/comment/j5wm0ix/)

ederuiter avatar Nov 18 '24 15:11 ederuiter

I also tested this without my changes and setting a breakpoint on the exact same spot:

gdb --args /home/eric/code/oss/chirpstack/target/debug/deps/chirpstack-5554738068c9e0c3 test::class_a_pr_test::test_sns_uplink
...
(gdb) b chirpstack::downlink::data::Data::set_mac_commands::{{closure}}
Breakpoint 1 at 0xd0622d: file chirpstack/src/downlink/data.rs, line 629.
(gdb) r
...
Switching to LWP 3410467]

Thread 2 "test::class_a_p" hit Breakpoint 1, chirpstack::downlink::data::Data::set_mac_commands::{{closure}} ()
    at chirpstack/src/downlink/data.rs:629
629	    async fn set_mac_commands(&mut self) -> Result<()> {
(gdb) bt
#0  chirpstack::downlink::data::Data::set_mac_commands::{{closure}} () at chirpstack/src/downlink/data.rs:629
...
#61 0x00007ffff7dd4dec in clone3 () from /nix/store/kpy2cyd05vdr6j1h200av81fnlxl1jw0-glibc-2.39-52/lib/libc.so.6
(gdb) frame 0
#0  chirpstack::downlink::data::Data::set_mac_commands::{{closure}} () at chirpstack/src/downlink/data.rs:629
629	    async fn set_mac_commands(&mut self) -> Result<()> {
(gdb) set $topsp=$sp
(gdb) frame 61
#61 0x00007ffff7dd4dec in clone3 () from /nix/store/kpy2cyd05vdr6j1h200av81fnlxl1jw0-glibc-2.39-52/lib/libc.so.6
(gdb) print $sp - $topsp
$1 = 1806416

So my changes only increased the used stack space by ~20KB (2094680 vs 1806416) .. but this was enough to push it over the edge

ederuiter avatar Nov 18 '24 16:11 ederuiter

I haven't had the time to look into this more deeply, but I found somewhere that there is a difference with regards to stack size when running tests (2MB) vs running the application (8MB). The fact that the tests are compile with debug profile doesn't help with regards to the stack size.

brocaar avatar Nov 19 '24 11:11 brocaar

Once the new version of deadpool-redis is released (https://github.com/bikeshedder/deadpool/pull/369) I will update this MR

However I did found out that redis-rs itself does not support active fail-over with sentinel :sob: (see https://github.com/redis-rs/redis-rs/issues/1438) It will select the right redis master on connect; however it will not migrate active connections to the new master once a fail-over event takes place. This means that with the current state of redis-rs; the sentinel support is not yet very helpful.

ederuiter avatar Dec 03 '24 11:12 ederuiter

Hi @ederuiter, would you mind rebasing the pull-request? Note that I just update redis-sentinel and redis crates in the master branch.

brocaar avatar Mar 20 '25 10:03 brocaar

Hi @ederuiter in case you missed my previous msg, I did update update the redis-deadpool and redis crates in the latest release. Could you update your PR?

brocaar avatar Apr 28 '25 09:04 brocaar