clickhouse-driver icon indicating copy to clipboard operation
clickhouse-driver copied to clipboard

Incorrect ping handling on connection close + reopen

Open sknaumov opened this issue 5 months ago • 0 comments

Describe the bug I'm using django-clickhouse-backend and want to work with clickhouse database using multiple python processes. For that purpose I need to close db connections before spawning new processes by calling

from django.db import connections

connections.close_all()

The problem is that if I've already queried DB in the main process before closing connections, I have the following warnings + exception:

clickhouse_driver.connection - WARNING - Error on clickhouse.***:9000 ping: Unexpected EOF while reading bytes
clickhouse_driver.connection - WARNING - Error on clickhouse.***:9000 ping: Unexpected EOF while reading bytes
...
clickhouse_driver.connection - WARNING - Connection was closed, reconnecting.
clickhouse_driver.connection - WARNING - Connection was closed, reconnecting.
...
django.db.utils.OperationalError: Code: 100. Unknown packet 4 from server clickhouse.***:9000

It seems that pings are not properly handled at connection reopening.

To Reproduce

Not a complete code, but general idea:

from datetime import datetime
from multiprocessing import Pool

from django.db import connections

def worker(interval):
    """get objects from clickhouse database for a given time interval"""
    
    objs = clickhouse_model.objects.filter(
        created_at__gte=interval.from,
        created_at__lt=interval.to,
    )

def main():
    """spawn multiple workers over time intervals"""

    min_date = clickhouse_model.objects.all.aggregate(min=Min('created_at'))['min']
    intervals = get_invervals(from=min_date, to=datetime.now(), size=86400)

    connections.close_all()  # Django will automatically reopen connection on the first query
    with Pool(processes=processes) as pool:
        for _ in pool.imap_unordered(worker, intervals):
             pass

Expected behavior

  • No warnings issued on user-initiated disconnect.
  • Pings are properly stopped.
  • I do not know why ping packets from closed connection are received on reopen, but they have to be silently discarded.

Versions

  • django-clickhouse-backend v1.3.0 + clickhouse-driver v0.2.8.
  • ClickHouse server v22.3.20.29.
  • Python v3.12.

sknaumov avatar Sep 18 '24 08:09 sknaumov