jaeger-clickhouse icon indicating copy to clipboard operation
jaeger-clickhouse copied to clipboard

[Feature]: Use native ClickHouse interface instead of database/sql

Open lodthe opened this issue 1 year ago • 0 comments

Requirement

As an active jaeger-clickhouse user I'd like to suggest to use the native ClickHouse communication protocol instead of database/sql-compatible one. This change might significantly increase the overall performance and speed up the spans writer.

Problem

jaeger-clickhouse uses the clickhouse-go 1.5.4 client. It provides the standard database/sql interface for communication with ClickHouse.

There is a benchmark section in the readme of the repository. It claims that migration to v2 might significantly speed up write and read. This speed up is possible due to usage of the native TCP ClickHouse client-server protocol. Furthermore, new versions (>= 2.3.0) use ch-go for compression.

Proposal

Switch to the newer version of the clickhouse-go client and enjoy the better performance.

Maybe it's even possible to switch to ch-go, but I think the library may not support all the used high-level features of clickhouse-go at the moment.

Open questions

I have two questions in mind:

  • Is it possible to switch to the native ClickHouse TCP protocol without breaking compatibility?
  • Is it worth it? We need to create a benchmark that compares two protocols on specific to jaeger-clickhouse queries.

Do you, folks, see any pitfalls?

lodthe avatar Jul 20 '22 12:07 lodthe