lmatz

Tracking: improve TPC-H performance

1

See performance numbers at https://www.notion.so/risingwave-labs/TPCH-Performance-Numbers-Table-e098ef82884546949333409f0513ada7?pvs=4#8de0bf4bda51444c8381f3b0c10ddfe1 - [ ] #15034 - [ ] #14811 - [ ] #15035 - [ ] #14799 - [ ] #14797

end-to-end test (release): `Error: snapshot id 4 not found`

3

### Describe the bug https://buildkite.com/risingwavelabs/main-cron/builds/1891#_ ``` 2024-02-23T16:23:34.102298669Z INFO risingwave_ctl::common::context: clean up context --   | Error: snapshot id 4 not found   | Backtrace:   | 0: msg   | at ./.cargo/registry/src/index.crates.io-6f17d22bba15001f/anyhow-1.0.75/src/error.rs:83:36   | 1:...

Chaos Mesh pod-kill-one-compute fails occasionally even the cluster seems to recover successfully

11

### Describe the bug https://buildkite.com/risingwave-test/longevity-chaos-mesh/builds/354#018c6cd1-1d73-460b-b33f-2a11377228a6 This pipeline's code itself is at: https://github.com/risingwavelabs/risingwave-test/pull/526 cc: @xuefengze if any question We notice that in this pipeline, we executed 12 tests for each nexmark...

> The eviction at the beginning of both tests is too aggressive. The eviction starts even when there is still quite much memory. first posted https://github.com/risingwavelabs/risingwave/issues/14797#issuecomment-1966549273 will collect a few...

perf: improve tpc-h q20 performance (single-topic)

14

See performance numbers at https://www.notion.so/risingwave-labs/TPCH-Performance-Numbers-Table-e098ef82884546949333409f0513ada7?pvs=4#8de0bf4bda51444c8381f3b0c10ddfe1

ch-benchmark q9 data verification fails in chaos mesh test

1

https://buildkite.com/risingwave-test/chaos-mesh/builds/590#018dcd0c-7f3b-4bb6-92ac-719ee95386d2 The experiments made the meta pod fail for 20 seconds

I imagine one approach is to put it into the log another one is to put it into a system table that can be queried. I feel the latter one...

perf: improve nexmark q15 baseline and scaling up performance

2

`nightly-20240224` RW Config: ``` RW_CONFIG="{'system':{'data_directory':'hummock_001','barrier_interval_ms':10000},'server':{'telemetry_enabled':false},'meta': {'level0_tier_compact_file_number':6,'level0_overlapping_sub_level_compact_level_count':6}}" ``` RW: 4X: http://metabase.risingwave-cloud.xyz/question/9236-nexmark-q15-blackhole-4x-medium-1cn-affinity-avg-source-output-rows-per-second-rows-s-history-thtb-2763?start_date=2024-01-04 https://grafana.test.risingwave-cloud.xyz/d/EpkBw5W4k/risingwave-dev-dashboard?orgId=1&var-datasource=Prometheus:%20test-useast1-eks-a&from=1708900035000&to=1708901778000&var-namespace=nexmark-ht-4x-1cn-affinity-10s https://buildkite.com/risingwave-test/nexmark-benchmark/builds/3156 1X: http://metabase.risingwave-cloud.xyz/question/603-nexmark-q15-blackhole-medium-1cn-affinity-avg-source-output-rows-per-second-rows-s-history-thtb-178?start_date=2023-11-17 https://grafana.test.risingwave-cloud.xyz/d/EpkBw5W4k/risingwave-dev-dashboard?orgId=1&var-datasource=Prometheus:%20test-useast1-eks-a&from=1708899363000&to=1708901166000&var-namespace=nexmark-1cn-affinity-10s https://buildkite.com/risingwave-test/nexmark-benchmark/builds/3155 RW 4X: 1.15M RW 1X: 546K 4X/1X Ratio: 2.1 Flink: 4X: http://metabase.risingwave-cloud.xyz/question/9712-flink-nexmark-q15-flink-4x-medium-1tm-avg-job-throughput-per-second-records-s-history-thtb-2920?start_date=2023-12-05 https://grafana.test.risingwave-cloud.xyz/d/Q4ayNkfVz/flink-hits?orgId=1&var-datasource=Prometheus:%20test-useast1-eks-a&from=1704381755000&to=1704382897000&var-namespace=flink-4x-medium-1tm-test-20240104 https://buildkite.com/risingwave-test/flink-nexmark-bench/builds/130...

Tracking: Introduce per-module error type for frontend

6

As mentioned in #1367. Tracking: - [x] #3247 - [x] https://github.com/singularity-data/risingwave/issues/2035 - [ ] Introduce HandleError Type - [ ] Introduce PlanError Type - [x] Introduce ScheduleError Type - [x]...

better query rewrite for TPC-H q17

1

Yes, I can not find a way to rewrite it... But it should ok for Q17 because the aggregator is an AVG, we should choose one to optimize #14799 -...