clickhouse-driver icon indicating copy to clipboard operation
clickhouse-driver copied to clipboard

Published 20 hours ago • verified publisher icon mymarilyn

clickhouse_driver.varint.read_varint is called recursively

Open vigneshshettyin opened this issue 1 year ago • 2 comments

Describe the bug Observed a case where when the query string is long, Package is taking more time to render data. Also ran a profiler and checked that database is returning result instantly (less than 500ms) but package is taking more 30-35 seconds.

To Reproduce Trigger API with higher query string (approx more than 575316)

Expected behavior Package should return response within 1 second

Versions

  • Version of package with the problem.
  • ClickHouse server version. Version can be obtained by running SELECT version() query. 23.12.1.956
  • Python version. 3.11

Please refer to https://github.com/jayvynl/django-clickhouse-backend/issues/70 as well

@jayvynl

vigneshshettyin avatar Dec 23 '23 05:12 vigneshshettyin

Hi.

Do you have any advices to achieve this speed? I've spent many hours on this thing and ended with pushing this code to cython. There are many places in binary protocol with numbers in LEB128. IMHO, this function is simple as it can be. The main time takes read_one function that reads one by one bytes from input stream.

xzkostyan avatar Dec 25 '23 19:12 xzkostyan

As of now, we don't have any solution. I am exploring stuff @xzkostyan

vigneshshettyin avatar Dec 26 '23 04:12 vigneshshettyin