Antonio Andelic
Antonio Andelic
@shiyer7474 you can clean all the files from disk/s3 before running the test
It should not, cleanup should make the Keeper start fresh from log idx 1
You can test your changes locally by running the test with `--count 2`
On each node you can remove all the files: ``` node1.exec_in_container(["bash", "-c", "rm -r /var/lib/clickhouse/coordination/log"]) node1.exec_in_container(["bash", "-c", "rm -r /var/lib/clickhouse/coordination/snapshots"]) ``` it's important to delete on all 3 nodes
You need to stop all instances before deleting the files to prevent the replication of the old data. After that you simply start them all.
can you please provide logs and/or full diff for the test?
because node1 is waiting synchronously for the other nodes, you need to start them all in async way ``` diff --git a/tests/integration/test_keeper_s3_snapshot/configs/keeper_config1.xml b/tests/integration/test_keeper_s3_snapshot/configs/keeper_config1.xml index 8459ea3e068..ce935739a5c 100644 --- a/tests/integration/test_keeper_s3_snapshot/configs/keeper_config1.xml +++ b/tests/integration/test_keeper_s3_snapshot/configs/keeper_config1.xml...
@serxa it's known that something strange is going on with Keeper after restarts during performance tests.
Flaky check failed because so many Keeper heavy tests were started on ASAN build.
https://s3.amazonaws.com/clickhouse-test-reports/66648/91e0df33dbfb81a788cfa69b6a0f8aac6e6a6f11/stateful_tests__tsan_.html ``` curl: (28) Operation timed out after 120001 milliseconds with 0 bytes received ``` It's also strange how there is 0 bytes received