clickhouse-java icon indicating copy to clipboard operation
clickhouse-java copied to clipboard

ZSTD compression

Open den-crane opened this issue 2 years ago • 0 comments

I would like to have ability to fetch results compressed by ZSTD

in my case ZSTD allows to fetch 3 times less data and twice faster:

### enable_http_compression=0

time curl "http://host1:8123/?enable_http_compression=0" -H 'Accept-Encoding: zstd' [email protected] -o http_no_compression
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  917M    0  917M  100 45885  51.0M   2550  0:00:17  0:00:17 --:--:-- 55.2M

real	0m17.996s


### compress=1 (LZ4)

time curl "http://host1:8123/?&compress=1" --data-binary @query.sql -o http_lz4
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  118M    0  118M  100 45885  29.3M  11371  0:00:04  0:00:04 --:--:-- 29.3M

real	0m4.089s


### enable_http_compression=1"  'Accept-Encoding: zstd'

time curl "http://host1:8123/?&enable_http_compression=1" -H 'Accept-Encoding: zstd' --data-binary @query.sql -o http_zstd3
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100 29.2M    0 29.1M  100 45885  10.8M  17025  0:00:02  0:00:02 --:--:-- 10.8M

real	0m2.713s


$ ping host1
...
64 bytes from 10.253.232.2 (10.253.232.2): icmp_seq=1 ttl=62 time=0.518 ms

1 GBit network

den-crane avatar Oct 03 '22 19:10 den-crane