clickhouse-java icon indicating copy to clipboard operation
clickhouse-java copied to clipboard

Fix wrong setting usage for creating output stream

Open exploder86 opened this issue 3 years ago • 4 comments

Value PERFORMANCE of settings "request_buffering" is ignored during output stream creation

exploder86 avatar Jul 14 '22 10:07 exploder86

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


anagaytsev seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.

CLAassistant avatar Jul 14 '22 10:07 CLAassistant

Benchmark                                (client)  (connection)  (statement)  (type)   Mode  Cnt     Score     Error  Units
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse       normal  object  thrpt   20   278.941 ±  45.219  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  object  thrpt   20   276.681 ±  20.723  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new       normal  object  thrpt   20   301.436 ±  30.832  ops/s
Basic.insertOneRandomNumber  clickhouse-http-jdbc           new     prepared  object  thrpt   20   276.675 ±  19.377  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20   307.368 ±  32.519  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20   272.618 ±  28.676  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new       normal  object  thrpt   20   309.456 ±  35.440  ops/s
Basic.insertOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  object  thrpt   20   271.317 ±  24.706  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse       normal  object  thrpt   20  1263.459 ±  89.564  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc         reuse     prepared  object  thrpt   20  1274.638 ± 101.754  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new       normal  object  thrpt   20  1245.750 ±  89.251  ops/s
Basic.selectOneRandomNumber  clickhouse-http-jdbc           new     prepared  object  thrpt   20  1308.418 ±  77.314  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse       normal  object  thrpt   20  1065.657 ±  98.016  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc         reuse     prepared  object  thrpt   20  1127.012 ±  88.512  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new       normal  object  thrpt   20  1102.328 ± 114.088  ops/s
Basic.selectOneRandomNumber  clickhouse-grpc-jdbc           new     prepared  object  thrpt   20  1156.629 ±  68.494  ops/s

github-actions[bot] avatar Jul 14 '22 10:07 github-actions[bot]

I found this bug when debugging clickhouse-jdbc, it hangs when executing big batch updates with spring jdbcTemplate. Batch size consists from 100 000 items and if buffer is small then main thread will loop inside NonBlockingPipedOutputStream#updateBuffer because timeout = 0 and CapacityPolicy also exceeded it's limits and return false.

exploder86 avatar Jul 14 '22 11:07 exploder86

I found this bug when debugging clickhouse-jdbc, it hangs when executing big batch updates with spring jdbcTemplate. Batch size consists from 100 000 items and if buffer is small then main thread will loop inside NonBlockingPipedOutputStream#updateBuffer because CapacityPolicy also exceeded it's limits and return false.

Thank you @exploder86, I think it's similar as #991.

zhicwu avatar Jul 14 '22 12:07 zhicwu