Memory Leak DynamicBitmap Index Scan in Nestloop's inner plan

[kainwen]

Bug Report

Greenplum version or build

Test under Master branch (top commit: ec10c4dfc45d204c4f383bd14b1fb6b50e2c2b23) and 6X_STABLE (top commit: 72d9dff1c40a2bdfae3cc3ddd79206da2a635298)

Reproduce SQL

NOTE: AO column is key to reproduce OOM. Under AO row, I do not reproduce OOM.

create table t1_12533 (a int, b int, c text, d int) with (appendonly = true, orientation = column)
distributed randomly
partition by range(a)
(
   start (1) end (1000) every (1),
   default partition extra
);
create table t2_12533 (a int, b int, c text, d int) with (appendonly = true, orientation = column)
distributed randomly
partition by range(a)
(
   start (1) end (1000) every (1),
   default partition extra
);

insert into t1_12533 select  i%1050, i, 'abcd'||i::text, i from generate_series(1, 10000)i;
insert into t2_12533 select * from t1_12533;
create index idx1_12533 ON t1_12533 USING bitmap (b);
create index idx2_12533 ON t2_12533 USING bitmap (b);

analyze t1_12533;
analyze t2_12533;
 
set optimizer_enable_hashjoin = off;
set optimizer_enable_materialize = off;
-- dynamic bitmapscan plan

select count(distinct a.d) from t1_12533 a, t2_12533 b
where a.c = b.c and a.b < 50 and b.b < 50;

Use htop to monitor QEs you will find the Mem keep increasing and finally this SQL fail due to OOM.

The plan is:

6X ORCA

 Aggregate
   ->  Gather Motion 3:1  (slice2; segments: 3)
         ->  Nested Loop
               Join Filter: true
               ->  Broadcast Motion 3:3  (slice1; segments: 3)
                     ->  Result
                           ->  Sequence
                                 ->  Partition Selector for t2_12533 (dynamic scan id: 2)
                                       Partitions selected: 1000 (out of 1000)
                                 ->  Dynamic Bitmap Heap Scan on t2_12533 (dynamic scan id: 2)
                                       Recheck Cond: (b < 600)
                                       ->  Dynamic Bitmap Index Scan on idx2_12533
                                             Index Cond: (b < 600)
               ->  Sequence
                     ->  Partition Selector for t1_12533 (dynamic scan id: 1)
                           Partitions selected: 1000 (out of 1000)
                     ->  Dynamic Bitmap Heap Scan on t1_12533 (dynamic scan id: 1)
                           Recheck Cond: (b < 600)
                           Filter: ((b < 600) AND (c = t2_12533.c))
                           ->  Dynamic Bitmap Index Scan on idx1_12533
                                 Index Cond: (b < 600)
 Optimizer: Pivotal Optimizer (GPORCA)

Master ORCA

 Finalize Aggregate
   ->  Gather Motion 2:1  (slice1; segments: 2)
         ->  Partial Aggregate
               ->  Redistribute Motion 2:2  (slice2; segments: 2)
                     Hash Key: t1_12533.d
                     ->  Nested Loop
                           Join Filter: true
                           ->  Broadcast Motion 2:2  (slice3; segments: 2)
                                 ->  Dynamic Seq Scan on t2_12533
                                       Number of partitions to scan: 1000
                                       Filter: (b < 600)
                           ->  Dynamic Bitmap Heap Scan on t1_12533
                                 Number of partitions to scan: 1000
                                 Recheck Cond: (b < 600)
                                 Filter: ((b < 600) AND (c = t2_12533.c))
                                 ->  Dynamic Bitmap Index Scan on idx1_12533
                                       Index Cond: (b < 600)
 Optimizer: Pivotal Optimizer (GPORCA)

Finally it will OOM, like:

ERROR:  Out of memory  (seg0 slice2 127.0.1.1:7002 pid=18632)
DETAIL:  System memory limit reached, failed to allocate 33008 bytes from system