Eric Fu

> Running a test with `stream_hash_agg_max_dirty_groups_heap_size` = 2MB > > https://buildkite.com/risingwave-test/ch-benchmark-pg-cdc/builds/104 > > I think it doesn't make sense that `stream_hash_agg_max_dirty_groups_heap_size` is much bigger than `mem_table_spill_threshold`. They should be same,...

Interesting observations on `stream_source_output_rows_counts` (total count, not `rate()`) ```promql stream_source_output_rows_counts{namespace=~"$namespace",risingwave_name=~"$instance",risingwave_component=~"$component",pod=~"$pod"} ``` `stream_source_output_rows_counts` of `stock` table `stream_source_output_rows_counts` of the rest tables The rest tables didn't consume any changes until 19:20. And...

There is a 1400x amplification in Q20. ```sql CREATE MATERIALIZED VIEW ch_benchmark_q20 AS SELECT s_name, s_address FROM supplier, nation WHERE s_suppkey IN ( SELECT mod(s_i_id * s_w_id, 10000) FROM stock,...

> > so each new line in order_line will be amplified 1400x > > Is big amplification supposed to be solved by spill anytime #12028? Partially. The big amplification may...

> However, I didn't find the heap dumps on s3 while the other two executions of the same pipeline today do exist. You probably need: ``` ENABLE_MEMORY_PROFILING="true" ``` This is...

I just analyzed [Run 149](https://buildkite.com/risingwave-test/ch-benchmark-pg-cdc/builds/149#018cd318-8708-418b-a709-ef40908c401d) (namespace: `ch-benchmark-pg-cdc-20240104-061254`) and the conclusion is same as above. The prost message decoding used more than 6GB memory. [1704349111-2024-01-04-06-18-30.auto.heap.collapsed.zip](https://github.com/risingwavelabs/risingwave/files/13827544/1704349111-2024-01-04-06-18-30.auto.heap.collapsed.zip)

Ping, Any updates?

> Currently our framework doesn't have a [checkpoint commit callback](https://github.com/apache/flink/blob/cf5bb80b5fd965928ff75d415dd93e0b12fa1b49/flink-core/src/main/java/org/apache/flink/api/common/state/CheckpointListener.java#L98) mechanism to notify the source executor. +1 for this. But that will certainly take some time. Before that, shall we...

Thanks a lot for the feedback! We'll consider it in roadmap

We do have an `NaN` in decimal https://github.com/risingwavelabs/risingwave/blob/ae7d7ba426818d43209be545ad72f979ce3dacc0/src/common/src/types/decimal.rs#L44