add IoT case for ClickHouse

[TCeason]

1. Add IoT case for ClickHouse

[CLAassistant]

All committers have signed the CLA.

[codecov-io]

Codecov Report

Merging #120 into master will increase coverage by 1.93%. The diff coverage is 94.57%.

@@            Coverage Diff             @@
##           master     #120      +/-   ##
==========================================
+ Coverage   69.59%   71.52%   +1.93%     
==========================================
  Files         104      105       +1     
  Lines        5812     6245     +433     
==========================================
+ Hits         4045     4467     +422     
- Misses       1689     1698       +9     
- Partials       78       80       +2     

Impacted Files	Coverage Δ
cmd/tsbs_load_clickhouse/process.go	2.01% <0.00%> (-0.03%)	:arrow_down:
internal/inputs/generator_queries.go	73.17% <ø> (ø)
internal/inputs/utils.go	78.26% <ø> (ø)
query/stat_processor.go	16.12% <0.00%> (ø)
cmd/tsbs_load_clickhouse/creator.go	36.47% <38.88%> (-0.01%)	:arrow_down:
...bs_generate_queries/databases/clickhouse/common.go	83.33% <75.00%> (-4.17%)	:arrow_down:
.../tsbs_generate_queries/databases/clickhouse/iot.go	100.00% <100.00%> (ø)
cmd/tsbs_load_clickhouse/main.go	47.72% <100.00%> (+2.48%)	:arrow_up:
query/benchmarker.go	75.78% <100.00%> (ø)
query/stats.go	100.00% <100.00%> (ø)
... and 1 more

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 9f1523d...b05b72b. Read the comment docs.

[den-crane]

ENGINE = MergeTree(created_date, (tags_id, created_at), 8192)

This is outdated syntax and this partitioning does not use partition pruning which makes part of queries much slower.

I recommend to replace it with

ENGINE = MergeTree 
Partition by toYYYYMMDD(created_at) 
Order by (tags_id,created_at) 
SETTINGS index_granularity=1024;

[TCeason]

ENGINE = MergeTree(created_date, (tags_id, created_at), 8192)

This is outdated syntax and this partitioning does not use partition pruning which makes part of queries much slower.

I recommend to replace it with

ENGINE = MergeTree 
Partition by toYYYYMMDD(created_at) 
Order by (tags_id,created_at) 
SETTINGS index_granularity=1024;

Ok

[RobAtticus]

@TCeason let us know when you'd like this looked at it. It appears to have conflicts at the moment.

[TCeason]

@TCeason let us know when you'd like this looked at it. It appears to have conflicts at the moment.

I see it and will fix it today.

[TCeason]

@RobAtticus waiting check.

https://github.com/timescale/tsbs/commit/270a6196abd87e5a19b5d4fb196fb05b55164f7f#diff-08d222aeabbb444fcb19d4df104ae679R21

[codecov-commenter]

Codecov Report

Merging #120 (1b8512c) into master (45b6321) will increase coverage by 2.13%. The diff coverage is 97.77%.

@@            Coverage Diff             @@
##           master     #120      +/-   ##
==========================================
+ Coverage   65.25%   67.38%   +2.13%     
==========================================
  Files         145      146       +1     
  Lines        5966     6363     +397     
==========================================
+ Hits         3893     4288     +395     
- Misses       1960     1961       +1     
- Partials      113      114       +1     

Impacted Files	Coverage Δ
pkg/query/stats.go	100.00% <ø> (ø)
pkg/targets/clickhouse/processor.go	1.66% <0.00%> (ø)
...bs_generate_queries/databases/clickhouse/common.go	80.00% <71.42%> (-4.62%)	:arrow_down:
.../tsbs_generate_queries/databases/clickhouse/iot.go	100.00% <100.00%> (ø)

---

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 45b6321...1b8512c. Read the comment docs.

[TCeason]

@den-crane Hi could this pr maybe be merged?

[Hadrien-Cornier]

Can someone resolve these conflicts and merge this ?

[TCeason]

@atanasovskib I have already resolved it.

[dssysolyatin]

When this PR will be merged ? No activity since March.

[TCeason]

When this PR will be merged ? No activity since March.

I also don't know.

[TCeason]

@den-crane Hi, Please review this pr and I have a question that who can merge this pr?

[jonatas]

Hello @TCeason! Thanks for rebasing and keeping it up to date!

I'll try to be testing it locally and reviewing it in the next few days! I already tested it locally but I'd like to double-check again after all the reviews!

[TCeason]

Hello @TCeason! Thanks for rebasing and keeping it up to date!

I'll try to be testing it locally and reviewing it in the next few days! I already tested it locally but I'd like to double-check again after all the reviews!

Ok, thx.

[TCeason]

@jonatas Hi, how's it going?

[TCeason]

scripts/full_cycle_minitest/full_cycle_minitest_clickhouse.sh

Ok, I will see it on weekend.

[TCeason]

My machine is Ubuntu 20.04 and Go version is go version go1.14.6 linux/amd64.

and I add some iot test cases. It can be run on my machine. Please review it.