greptimedb
greptimedb copied to clipboard
Published
20 hours ago •
GreptimeTeam
GreptimeTeam
Test mysql::mysql_server_test::test_server_require_secure_client_secure fails
What type of bug is this?
Unexpected error
What subsystems are affected?
Datanode, Other
What happened?
mysql::mysql_server_test::test_server_require_secure_client_secure ---
thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Io(Custom { kind: UnexpectedEof, error: "tls handshake eof" }))', src/servers/tests/mysql/mysql_server_test.rs:219:10
stack backtrace:
0: rust_begin_unwind
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:584:5
1: core::panicking::panic_fmt
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panicking.rs:142:14
2: core::result::unwrap_failed
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/result.rs:1805:5
3: <core::result::Result<mysql_async::conn::Conn, mysql_async::error::Error>>::unwrap
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/result.rs:1098:23
4: mod::mysql::mysql_server_test::do_test_query_all_datatypes::{closure#0}
at ./tests/mysql/mysql_server_test.rs:217:26
5: <core::future::from_generator::GenFuture<mod::mysql::mysql_server_test::do_test_query_all_datatypes::{closure#0}> as core::future::future::Future>::poll
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/future/mod.rs:91:19
6: mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{closure#0}
at ./tests/mysql/mysql_server_test.rs:161:63
7: <core::future::from_generator::GenFuture<mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{closure#0}> as core::future::future::Future>::poll
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/future/mod.rs:91:19
8: <tokio::park::thread::CachedParkThread>::block_on::<core::future::from_generator::GenFuture<mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{closure#0}>>::{closure#0}
at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/park/thread.rs:267:54
9: tokio::coop::with_budget::<core::task::poll::Poll<core::result::Result<(), servers::error::Error>>, <tokio::park::thread::CachedParkThread>::block_on<core::future::from_generator::GenFuture<mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{closure#0}>>::{closure#0}>::{closure#0}
at /home/runner/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:102:9
10: <std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>>::try_with::<tokio::coop::with_budget<core::task::poll::Poll<core::result::Result<(), servers::error::Error>>, <tokio::park::thread::CachedParkThread>::block_on<core::future::from_generator::GenFuture<mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{closure#0}>>::{closure#0}>::{closure#0}, core::task::poll::Poll<core::result::Result<(), servers::error::Error>>>
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/local.rs:445:16
11: <std::thread::local::LocalKey<core::cell::Cell<tokio::coop::Budget>>>::with::<tokio::coop::with_budget<core::task::poll::Poll<core::result::Result<(), servers::error::Error>>, <tokio::park::thread::CachedParkThread>::block_on<core::future::from_generator::GenFuture<mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{closure#0}>>::{closure#0}>::{closure#0}, core::task::poll::Poll<core::result::Result<(), servers::error::Error>>>
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/local.rs:421:9
12: tokio::coop::with_budget::<core::task::poll::Poll<core::result::Result<(), servers::error::Error>>,
What operating system did you use?
Ubuntu 21.04 x64,
Relevant log output and stack trace
No response
How can we reproduce the bug?
Run the test, it's not constant.
Complete backtrace and log:
2022-12-02T06:14:47.655187Z ERROR servers::mysql::server: Internal error occurred during query exec, server actively close the channel to let client try next time. err.msg=Internal IO error, source: received corrupt message err.code=Internal err.source=received corrupt message
test mysql::mysql_server_test::test_server_require_secure_client_secure ... FAILED
failures:
failures:
mysql::mysql_server_test::test_server_require_secure_client_secure
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 29 filtered out; finished in 0.14s
--- STDERR: servers::mod mysql::mysql_server_test::test_server_require_secure_client_secure ---
thread 'mysql::mysql_server_test::test_server_require_secure_client_secure' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Io(Custom { kind: UnexpectedEof, error: "tls handshake eof" }))', src/servers/tests/mysql/mysql_server_test.rs:219:10
stack backtrace:
0: 0x107405fa0 - std::backtrace_rs::backtrace::libunwind::trace::hc14e6571692321f9
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
1: 0x107405fa0 - std::backtrace_rs::backtrace::trace_unsynchronized::ha925ef12400cb0e8
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x107405fa0 - std::sys_common::backtrace::_print_fmt::hca710ff42dd9464b
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:66:5
3: 0x107405fa0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hd978193aa12372ea
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:45:22
4: 0x107427624 - core::fmt::write::hd2acea10475a590a
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/fmt/mod.rs:1198:17
5: 0x1073ff8fc - std::io::Write::write_fmt::hf32a0b04001073b3
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/io/mod.rs:1672:15
6: 0x1074077f4 - std::sys_common::backtrace::_print::hecc0e5125f4c902e
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:48:5
7: 0x1074077f4 - std::sys_common::backtrace::print::h340218e849051a6e
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:35:9
8: 0x1074077f4 - std::panicking::default_hook::{{closure}}::h999216a9d686b3d1
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:295:22
9: 0x107407518 - std::panicking::default_hook::hb92d84420f9fe964
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:314:9
10: 0x107407d8c - std::panicking::rust_panic_with_hook::hfec3a817f28b6272
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:698:17
11: 0x107407cc0 - std::panicking::begin_panic_handler::{{closure}}::hc0e1005d0448d503
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:588:13
12: 0x1074064ac - std::sys_common::backtrace::__rust_end_short_backtrace::hbacb9052f7488bf6
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:138:18
13: 0x107407a18 - rust_begin_unwind
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:584:5
14: 0x1074a5a9c - core::panicking::panic_fmt::hb18467e0597b232a
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panicking.rs:142:14
15: 0x1074a5b88 - core::result::unwrap_failed::he1ebe8715ad8c586
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/result.rs:1805:5
16: 0x102fb2908 - core::result::Result<T,E>::unwrap::h28ae318b703ca395
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/result.rs:1098:23
17: 0x10309d45c - mod::mysql::mysql_server_test::do_test_query_all_datatypes::{{closure}}::ha539884fd04fed8f
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:217:26
18: 0x102fab220 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::hcd054a4b00a29c6b
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/future/mod.rs:91:19
19: 0x1030a0ec8 - mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{{closure}}::hbb4e1028fa1a9031
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:161:63
20: 0x102fa911c - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h7b1ac2048e6a31d0
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/future/mod.rs:91:19
21: 0x102f9e6f0 - tokio::park::thread::CachedParkThread::block_on::{{closure}}::h40fa7e618b31b334
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/park/thread.rs:267:54
22: 0x1031013d8 - tokio::coop::with_budget::{{closure}}::h820578e172878eb2
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:102:9
23: 0x102fc91f4 - std::thread::local::LocalKey<T>::try_with::h1052a2440dc03f42
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/local.rs:445:16
24: 0x102fc8864 - std::thread::local::LocalKey<T>::with::hf78f6ec9ffa7ab75
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/local.rs:421:9
25: 0x102f940d0 - tokio::coop::with_budget::h9c8ffdd20b6896e9
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:95:5
26: 0x102f940d0 - tokio::coop::budget::h85bd4bc7f88c8b4e
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:72:5
27: 0x102f940d0 - tokio::park::thread::CachedParkThread::block_on::h1add066efbf639f5
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/park/thread.rs:267:31
28: 0x102fc060c - tokio::runtime::enter::Enter::block_on::ha278350cc537f66c
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/runtime/enter.rs:152:13
29: 0x103066878 - tokio::runtime::scheduler::multi_thread::MultiThread::block_on::hfad1167dfe5c2e69
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/runtime/scheduler/multi_thread/mod.rs:79:9
30: 0x10314c7c0 - tokio::runtime::Runtime::block_on::h7574f04bac367e09
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/runtime/mod.rs:492:44
31: 0x103007764 - mod::mysql::mysql_server_test::test_server_require_secure_client_secure::hde9bb29b8b9f99d9
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:162:5
32: 0x1030a0c8c - mod::mysql::mysql_server_test::test_server_require_secure_client_secure::{{closure}}::h76ea924de0cdd1c4
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:153:7
33: 0x10301a07c - core::ops::function::FnOnce::call_once::h70e36718ba7eef64
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/ops/function.rs:248:5
34: 0x10317c3e0 - core::ops::function::FnOnce::call_once::h448355ad81ee5104
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/ops/function.rs:248:5
35: 0x10317c3e0 - test::__rust_begin_short_backtrace::h1e3199ceedceead7
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:572:5
36: 0x10317b628 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hac7676a220744ac4
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/alloc/src/boxed.rs:1934:9
37: 0x10317b628 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hc1c0c9a59b8f8797
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panic/unwind_safe.rs:271:9
38: 0x10317b628 - std::panicking::try::do_call::h4ac749b92bb1de26
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:492:40
39: 0x10317b628 - std::panicking::try::hd0923b4c454edbdc
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:456:19
40: 0x10317b628 - std::panic::catch_unwind::h1a3a2d07675f42b5
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panic.rs:137:14
41: 0x10317b628 - test::run_test_in_process::h21aa03a39d2a5fc3
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:595:18
42: 0x10317b628 - test::run_test::run_test_inner::{{closure}}::ha5695eb2af6683bd
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:489:39
43: 0x10314e13c - test::run_test::run_test_inner::{{closure}}::h159ca882096a440b
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:516:37
44: 0x10314e13c - std::sys_common::backtrace::__rust_begin_short_backtrace::hb6a883f65e15d246
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:122:18
45: 0x10315457c - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h78a5f792d13f2b3e
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/mod.rs:505:17
46: 0x10315457c - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h50d36478a764bec4
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panic/unwind_safe.rs:271:9
47: 0x10315457c - std::panicking::try::do_call::h38b4164589e33f31
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:492:40
48: 0x10315457c - std::panicking::try::hf02d894065dfc68c
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:456:19
49: 0x10315457c - std::panic::catch_unwind::h26b3db2fd141f6c9
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panic.rs:137:14
50: 0x10315457c - std::thread::Builder::spawn_unchecked_::{{closure}}::hca5d3700095e879c
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/mod.rs:504:30
51: 0x10315457c - core::ops::function::FnOnce::call_once{{vtable.shim}}::hc397f906f3f1fb32
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/ops/function.rs:248:5
52: 0x10740cc54 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hdf963a76665f0880
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/alloc/src/boxed.rs:1934:9
53: 0x10740cc54 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h58cf84b849fcaf60
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/alloc/src/boxed.rs:1934:9
54: 0x10740cc54 - std::sys::unix::thread::Thread::new::thread_start::h6fb3ec738344ccb5
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys/unix/thread.rs:108:17
55: 0x1b666c26c - __pthread_deallocate
Canceling due to test failure: 9 tests still running
FAIL [ 0.146s] servers::mod mysql::mysql_server_test::test_server_prefer_secure_client_secure
--- STDOUT: servers::mod mysql::mysql_server_test::test_server_prefer_secure_client_secure ---
running 1 test
2022-12-02T06:14:47.650068Z INFO common_telemetry::logging: logs dir = /tmp/__unittest_logs
2022-12-02T06:14:47.650842Z INFO servers::server: MySQL server started at 127.0.0.1:56651
2022-12-02T06:14:47.651532Z INFO servers::mysql::server: MySQL connection coming from: 127.0.0.1:56653
2022-12-02T06:14:47.654661Z DEBUG rustls::client::hs: No cached session for IpAddress(127.0.0.1)
2022-12-02T06:14:47.654810Z DEBUG rustls::client::hs: Not resuming any session
2022-12-02T06:14:47.655042Z ERROR servers::mysql::server: Internal error occurred during query exec, server actively close the channel to let client try next time. err.msg=Internal IO error, source: received corrupt message err.code=Internal err.source=received corrupt message
test mysql::mysql_server_test::test_server_prefer_secure_client_secure ... FAILED
failures:
failures:
mysql::mysql_server_test::test_server_prefer_secure_client_secure
test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 29 filtered out; finished in 0.14s
--- STDERR: servers::mod mysql::mysql_server_test::test_server_prefer_secure_client_secure ---
thread 'mysql::mysql_server_test::test_server_prefer_secure_client_secure' panicked at 'called `Result::unwrap()` on an `Err` value: Io(Io(Custom { kind: UnexpectedEof, error: "tls handshake eof" }))', src/servers/tests/mysql/mysql_server_test.rs:219:10
stack backtrace:
0: 0x105415fa0 - std::backtrace_rs::backtrace::libunwind::trace::hc14e6571692321f9
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
1: 0x105415fa0 - std::backtrace_rs::backtrace::trace_unsynchronized::ha925ef12400cb0e8
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/../../backtrace/src/backtrace/mod.rs:66:5
2: 0x105415fa0 - std::sys_common::backtrace::_print_fmt::hca710ff42dd9464b
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:66:5
3: 0x105415fa0 - <std::sys_common::backtrace::_print::DisplayBacktrace as core::fmt::Display>::fmt::hd978193aa12372ea
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:45:22
4: 0x105437624 - core::fmt::write::hd2acea10475a590a
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/fmt/mod.rs:1198:17
5: 0x10540f8fc - std::io::Write::write_fmt::hf32a0b04001073b3
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/io/mod.rs:1672:15
6: 0x1054177f4 - std::sys_common::backtrace::_print::hecc0e5125f4c902e
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:48:5
7: 0x1054177f4 - std::sys_common::backtrace::print::h340218e849051a6e
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:35:9
8: 0x1054177f4 - std::panicking::default_hook::{{closure}}::h999216a9d686b3d1
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:295:22
9: 0x105417518 - std::panicking::default_hook::hb92d84420f9fe964
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:314:9
10: 0x105417d8c - std::panicking::rust_panic_with_hook::hfec3a817f28b6272
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:698:17
11: 0x105417cc0 - std::panicking::begin_panic_handler::{{closure}}::hc0e1005d0448d503
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:588:13
12: 0x1054164ac - std::sys_common::backtrace::__rust_end_short_backtrace::hbacb9052f7488bf6
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:138:18
13: 0x105417a18 - rust_begin_unwind
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:584:5
14: 0x1054b5a9c - core::panicking::panic_fmt::hb18467e0597b232a
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panicking.rs:142:14
15: 0x1054b5b88 - core::result::unwrap_failed::he1ebe8715ad8c586
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/result.rs:1805:5
16: 0x100fc2908 - core::result::Result<T,E>::unwrap::h28ae318b703ca395
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/result.rs:1098:23
17: 0x1010ad45c - mod::mysql::mysql_server_test::do_test_query_all_datatypes::{{closure}}::ha539884fd04fed8f
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:217:26
18: 0x100fbb220 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::hcd054a4b00a29c6b
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/future/mod.rs:91:19
19: 0x1010b0b00 - mod::mysql::mysql_server_test::test_server_prefer_secure_client_secure::{{closure}}::hca85ee79b09f7311
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:148:63
20: 0x100fb8708 - <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll::h64aedbffb4169c91
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/future/mod.rs:91:19
21: 0x100faecb8 - tokio::park::thread::CachedParkThread::block_on::{{closure}}::h7960e07e755a4811
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/park/thread.rs:267:54
22: 0x10110d730 - tokio::coop::with_budget::{{closure}}::h033dd09fb53d898f
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:102:9
23: 0x100fe0c9c - std::thread::local::LocalKey<T>::try_with::h9a0486767b72a92f
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/local.rs:445:16
24: 0x100fd7dd0 - std::thread::local::LocalKey<T>::with::hd3736ca8419b4f24
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/local.rs:421:9
25: 0x100fab870 - tokio::coop::with_budget::h49bd200c078804d2
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:95:5
26: 0x100fab870 - tokio::coop::budget::h4804dd67e21df546
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/coop.rs:72:5
27: 0x100fab870 - tokio::park::thread::CachedParkThread::block_on::hbf9784eeeaf2f082
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/park/thread.rs:267:31
28: 0x100fd0d80 - tokio::runtime::enter::Enter::block_on::he24233bb822c9ffc
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/runtime/enter.rs:152:13
29: 0x101075d94 - tokio::runtime::scheduler::multi_thread::MultiThread::block_on::h9771c247b697c83f
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/runtime/scheduler/multi_thread/mod.rs:79:9
30: 0x10115b758 - tokio::runtime::Runtime::block_on::h22d8b9fd6b1f5a02
at /Users/dennis/.cargo/registry/src/github.com-1ecc6299db9ec823/tokio-1.21.1/src/runtime/mod.rs:492:44
31: 0x1010175ec - mod::mysql::mysql_server_test::test_server_prefer_secure_client_secure::h639a2a9b32342c58
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:149:5
32: 0x1010b08c4 - mod::mysql::mysql_server_test::test_server_prefer_secure_client_secure::{{closure}}::h5653fe016c7f5430
at /Users/dennis/programming/rust/greptimedb/src/servers/tests/mysql/mysql_server_test.rs:140:7
33: 0x10102a634 - core::ops::function::FnOnce::call_once::h91b0c104c17e91ce
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/ops/function.rs:248:5
34: 0x10118c3e0 - core::ops::function::FnOnce::call_once::h448355ad81ee5104
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/ops/function.rs:248:5
35: 0x10118c3e0 - test::__rust_begin_short_backtrace::h1e3199ceedceead7
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:572:5
36: 0x10118b628 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hac7676a220744ac4
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/alloc/src/boxed.rs:1934:9
37: 0x10118b628 - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::hc1c0c9a59b8f8797
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panic/unwind_safe.rs:271:9
38: 0x10118b628 - std::panicking::try::do_call::h4ac749b92bb1de26
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:492:40
39: 0x10118b628 - std::panicking::try::hd0923b4c454edbdc
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:456:19
40: 0x10118b628 - std::panic::catch_unwind::h1a3a2d07675f42b5
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panic.rs:137:14
41: 0x10118b628 - test::run_test_in_process::h21aa03a39d2a5fc3
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:595:18
42: 0x10118b628 - test::run_test::run_test_inner::{{closure}}::ha5695eb2af6683bd
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:489:39
43: 0x10115e13c - test::run_test::run_test_inner::{{closure}}::h159ca882096a440b
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/test/src/lib.rs:516:37
44: 0x10115e13c - std::sys_common::backtrace::__rust_begin_short_backtrace::hb6a883f65e15d246
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys_common/backtrace.rs:122:18
45: 0x10116457c - std::thread::Builder::spawn_unchecked_::{{closure}}::{{closure}}::h78a5f792d13f2b3e
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/mod.rs:505:17
46: 0x10116457c - <core::panic::unwind_safe::AssertUnwindSafe<F> as core::ops::function::FnOnce<()>>::call_once::h50d36478a764bec4
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/panic/unwind_safe.rs:271:9
47: 0x10116457c - std::panicking::try::do_call::h38b4164589e33f31
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:492:40
48: 0x10116457c - std::panicking::try::hf02d894065dfc68c
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panicking.rs:456:19
49: 0x10116457c - std::panic::catch_unwind::h26b3db2fd141f6c9
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/panic.rs:137:14
50: 0x10116457c - std::thread::Builder::spawn_unchecked_::{{closure}}::hca5d3700095e879c
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/thread/mod.rs:504:30
51: 0x10116457c - core::ops::function::FnOnce::call_once{{vtable.shim}}::hc397f906f3f1fb32
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/core/src/ops/function.rs:248:5
52: 0x10541cc54 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::hdf963a76665f0880
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/alloc/src/boxed.rs:1934:9
53: 0x10541cc54 - <alloc::boxed::Box<F,A> as core::ops::function::FnOnce<Args>>::call_once::h58cf84b849fcaf60
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/alloc/src/boxed.rs:1934:9
54: 0x10541cc54 - std::sys::unix::thread::Thread::new::thread_start::h6fb3ec738344ccb5
at /rustc/87588a2afd9ca903366f0deaf84d805f34469384/library/std/src/sys/unix/thread.rs:108:17
55: 0x1b666c26c - __pthread_deallocate
It seems that the tokio test concurrency setting related. It was always "tls handshake eof when I test this case with #[tokio::test] setting until I change this to #[tokio::test(flavor = "multi_thread", worker_threads = 2)] .
I make a PR to change worker_threads to 4. https://github.com/GreptimeTeam/greptimedb/pull/701
@SSebo Thanks. But we still don't know the root cause, let's keep this issue open until we find out the cause.
This EOF error happens quite frequently in our cloud deployment. It should be a bug in the protocol layer.
