cloudberry
cloudberry copied to clipboard
Pax/support bloom filter pushdown
this pr is base on https://github.com/apache/cloudberry/pull/1324
feat: pushdown bloom filter to pax table am
This optimization pushes down Bloom Filter conditions for runtime filters to the Pax Table AM layer.
By applying the filter earlier than the SeqNext() function, it eliminates the overhead of converting data from columnar format to TableTupleSlot, resulting in faster query execution
CREATE TABLE t2(c1 int, c2 int, c3 int, c4 int, c5 int) with (appendonly=true, orientation=column) distributed REPLICATED;
INSERT INTO t2 VALUES (1,1,1,1,1), (2,2,2,2,2), (3,3,3,3,3), (4,4,4,4,4);
INSERT INTO t2 select * FROM t2;
INSERT INTO t2 select * FROM t2;
INSERT INTO t2 select * FROM t2;
CREATE TABLE t3(c1 int, c2 int, c3 int, c4 int, c5 int) using pax;
gpadmin=# insert into t3 select b,i,i,i,i from generate_series(1,5) b,generate_series(1,2000000) i;
INSERT 0 10000000
gpadmin=#
gpadmin=# analyze t3;
ANALYZE
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
QUERY PLAN
-------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (actual time=355.000..1482.000 rows=160 loops=1)
-> Hash Join (actual time=0.000..1482.000 rows=96 loops=1)
Hash Cond: (t3.c2 = t2.c4)
Extra Text: (seg0) Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
-> Seq Scan on t3 (actual time=0.000..691.000 rows=6000000 loops=1)
-> Hash (actual time=0.000..0.000 rows=32 loops=1)
Buckets: 524288 Batches: 1 Memory Usage: 4098kB
-> Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
Optimizer: GPORCA
(9 rows)
gpadmin=# set gp_enable_runtime_filter_pushdown to on;
SET
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
QUERY PLAN
-------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (actual time=99.000..303.000 rows=160 loops=1)
-> Hash Join (actual time=1.000..303.000 rows=96 loops=1)
Hash Cond: (t3.c2 = t2.c4)
Extra Text: (seg0) Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
-> Seq Scan on t3 (actual time=1.000..301.000 rows=12 loops=1)
Rows Removed by Pushdown Runtime Filter: 5999988
-> Hash (actual time=0.000..0.000 rows=32 loops=1)
Buckets: 524288 Batches: 1 Memory Usage: 4098kB
-> Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
Optimizer: GPORCA
(10 rows)
gpadmin=# set pax_enable_row_filter to on;
SET
gpadmin=# EXPLAIN (ANALYZE, COSTS OFF, SUMMARY OFF)
SELECT t3.c3 FROM t3, t2 WHERE t3.c2 = t2.c4;
QUERY PLAN
-------------------------------------------------------------------------------------------
Gather Motion 3:1 (slice1; segments: 3) (actual time=48.000..138.000 rows=160 loops=1)
-> Hash Join (actual time=0.000..138.000 rows=96 loops=1)
Hash Cond: (t3.c2 = t2.c4)
Extra Text: (seg0) Hash chain length 8.0 avg, 8 max, using 4 of 524288 buckets.
-> Seq Scan on t3 (actual time=0.000..137.000 rows=12 loops=1)
Rows Removed by Pushdown Runtime Filter: 5999988
-> Hash (actual time=0.000..0.000 rows=32 loops=1)
Buckets: 524288 Batches: 1 Memory Usage: 4098kB
-> Seq Scan on t2 (actual time=0.000..0.000 rows=32 loops=1)
Optimizer: GPORCA
(10 rows)
What does this PR do?
Type of Change
- [ ] Bug fix (non-breaking change)
- [ ] New feature (non-breaking change)
- [ ] Breaking change (fix or feature with breaking changes)
- [ ] Documentation update
Breaking Changes
Test Plan
- [ ] Unit tests added/updated
- [ ] Integration tests added/updated
- [ ] Passed
make installcheck - [ ] Passed
make -C src/test installcheck-cbdb-parallel
Impact
Performance:
User-facing changes:
Dependencies:
Checklist
- [ ] Followed contribution guide
- [ ] Added/updated documentation
- [ ] Reviewed code for security implications
- [ ] Requested review from cloudberry committers