Kamil Braun

Could be that there's some retry going on behind the scenes, which causes the ALTER to be executed twice? I see the ALTER happens concurrently with one node being killed....

Looking at the logs, the driver retry hypothesis looks likely. In [this run](https://github.com/scylladb/scylladb/issues/17362#issuecomment-1963057137) I see the following: on node 2, the one which is killed by the test, we have:...

Python driver code cassandra/policies.py ```python3 class RetryPolicy(object): ... def on_request_error(self, query, consistency, error, retry_num): """ This is called when an unexpected error happens. This can be in the following situations:...

I see the test was flaky before: https://github.com/scylladb/scylladb/issues/15200 but in a different way --- the query was timing out. We made several improvements for schema changes and query liveness in...

cc @fruch @temichus see above the test is flaky by design, after several driver improvements it just changed how the flakiness shows. We need to either take it out of...

> expect failures, you also mean retrying them ? cause this test is setup there there a function (create_table, alter, drop) that it would run during a node being stopped....

node3 is stopping gracefully, so in theory there should be a notification to other nodes that it's going down, thus there should be no race. But I don't remember if...

No idea. Please provide full log from node4 as a file.

@rohitraj-carousellgroup please do the following: - shutdown the node that you tried to add (node 4) and remove its work directories (data, commitlog etc. -- the entire workdir) - follow...

@rohitraj-carousellgroup please provide the output of ``` select * from system.raft_state ``` executed on every node in the cluster