clickhouse-driver
clickhouse-driver copied to clipboard
Inserted rows disappear even though insert is successful
Hi!
We have a CH cluster with 3 replicas. Occasionally (maybe once a day), we're replacing the data in a table with new data. For that we do:
clickhouse_client.execute("ALTER TABLE target_table DELETE WHERE bh_client_id = 1492 SETTINGS mutations_sync = 2")
# at this point, because of mutations_sync = 2, the delete operation should be completed on all replicas
for records in batches:
inserted = clickhouse_client.execute('INSERT INTO target_table VALUES', records)
assert len(records) == inserted
Problem is, even though the INSERT command reports that all records were inserted, quite often (around 50% of the times), if we inspect target_tabe
for records with bh_client_id = 1492
. We don't find any!
We're really puzzled by this error, we're no CH experts and any help or hint on what might be causing it would be greatly appreciated! 🙏
insert_quorum
is set to 0.
One thing we do see in the server's error logs is the following:
2023.07.20 08:01:05.703697 [ 2318552 ] {} <Warning> HTTPServerRequest: Got an HTTP request with no content length and no chunked/multipart encoding, it may be impossible to distinguish graceful EOF from abnormal connection loss
I don't know if this is an issue with the driver not sending a proper HTTP request.
Versions
- Package: clickhouse-driver[lz4]==0.2.6.
- ClickHouse server version: 23.2.4.12
- Python version: 3.9.10