UT test_db_write_buffer_size_mem_wal failed sometimes

[jiacai2050]

Describe this problem

---- tests::read_write_test::test_db_write_buffer_size_mem_wal stdout ----
thread 'tests::read_write_test::test_db_write_buffer_size_mem_wal' panicked at analytic_engine/src/tests/read_write_test.rs:677:9:
assertion `left == right` failed
  left: 1
 right: 0
stack backtrace:
   0: rust_begin_unwind
             at /rustc/8550f15e148407159af401e02b1d9259762b3496/library/std/src/panicking.rs:617:5
   1: core::panicking::panic_fmt
             at /rustc/8550f15e148407159af401e02b1d9259762b3496/library/core/src/panicking.rs:67:14
   2: core::panicking::assert_failed_inner
   3: core::panicking::assert_failed
             at /rustc/8550f15e148407159af401e02b1d9259762b3496/library/core/src/panicking.rs:248:5
   4: analytic_engine::tests::read_write_test::test_write_buffer_size_overflow::{{closure}}
             at ./src/tests/read_write_test.rs:677:9
   5: tokio::runtime::park::CachedParkThread::block_on::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/park.rs:283:63
   6: tokio::runtime::coop::with_budget
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/coop.rs:107:5
   7: tokio::runtime::coop::budget
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/coop.rs:73:5
   8: tokio::runtime::park::CachedParkThread::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/park.rs:283:31
   9: tokio::runtime::context::blocking::BlockingRegionGuard::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/context/blocking.rs:66:9
  10: tokio::runtime::scheduler::multi_thread::MultiThread::block_on::{{closure}}
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/scheduler/multi_thread/mod.rs:87:13
  11: tokio::runtime::context::runtime::enter_runtime
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/context/runtime.rs:65:16
  12: tokio::runtime::scheduler::multi_thread::MultiThread::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/scheduler/multi_thread/mod.rs:86:9
  13: tokio::runtime::runtime::Runtime::block_on
             at /home/runner/.cargo/registry/src/index.crates.io-6f17d22bba15001f/tokio-1.29.1/src/runtime/runtime.rs:313:45
  14: runtime::Runtime::block_on
             at /home/runner/work/ceresdb/ceresdb/components/runtime/src/lib.rs:98:9
  15: analytic_engine::tests::util::TestEnv::block_on
             at ./src/tests/util.rs:472:9
  16: analytic_engine::tests::read_write_test::test_write_buffer_size_overflow
             at ./src/tests/read_write_test.rs:587:5
  17: analytic_engine::tests::read_write_test::test_db_write_buffer_size
             at ./src/tests/read_write_test.rs:554:5
  18: analytic_engine::tests::read_write_test::test_db_write_buffer_size_mem_wal
             at ./src/tests/read_write_test.rs:546:9
  19: analytic_engine::tests::read_write_test::test_db_write_buffer_size_mem_wal::{{closure}}
             at ./src/tests/read_write_test.rs:542:40
  20: core::ops::function::FnOnce::call_once
             at /rustc/8550f15e148407159af401e02b1d9259[762](https://github.com/CeresDB/ceresdb/actions/runs/6439577851/job/17487413595?pr=1240#step:8:763)b3496/library/core/src/ops/function.rs:250:5
  21: core::ops::function::FnOnce::call_once
             at /rustc/8550f15e148407159af401e02b1d9259762b3496/library/core/src/ops/function.rs:250:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.


failures:
    tests::read_write_test::test_db_write_buffer_size_mem_wal

test result: FAILED. 92 passed; 1 failed; 2 ignored; 0 measured; 0 filtered out; finished in 9.89s

error: test failed, to rerun pass `-p analytic_engine --lib`
make: *** [Makefile:54: test-ut] Error 101
Error: Process completed with exit code 2.

Server version

https://github.com/CeresDB/ceresdb/actions/runs/6439577851/job/17487413595?pr=1240

Steps to reproduce

Difficult to reproduce, this happens sometimes in CI.

Expected behavior

No error

Additional Information

No response

[jiacai2050]

Sadly, this still happens frequently in CI...