citus icon indicating copy to clipboard operation
citus copied to clipboard

some parts of create_distributed_table_concurrently ignores max_adaptive_executor_pool_size

Open onderkalaci opened this issue 3 years ago • 1 comments

drop table if exists ref;
create table ref (id int primary key);
select create_reference_table('ref');
insert into ref select s from generate_series(0,9) s;
drop table if exists test_1, test_2, test;
create table test (x int, y int references ref (id), t timestamptz default now()) partition by range (t);
create table test_1 partition of test for values from ('2022-01-01') to ('2022-12-31');
create table test_2 partition of test for values from ('2023-01-01') to ('2023-12-31'); 
insert into test (x,y) select s,s%10 from generate_series(1,100) s;
set citus.log_remote_commands to on;


SET citus.max_adaptive_executor_pool_size TO 1;
select create_distributed_table_concurrently('test','x');

And, concurrently check

 select * from citus_remote_connection_stats();

┌───────────┬──────┬───────────────┬──────────────────────────┐
│ hostname  │ port │ database_name │ connection_count_to_node │
├───────────┼──────┼───────────────┼──────────────────────────┤
│ localhost │ 9706 │ postgres      │                       12 │
│ localhost │ 9701 │ postgres      │                       11 │
│ localhost │ 9703 │ postgres      │                       13 │
│ localhost │ 9705 │ postgres      │                        9 │
│ localhost │ 9702 │ postgres      │                       14 │
│ localhost │ 9700 │ postgres      │                       16 │
│ localhost │ 5432 │ postgres      │                        4 │
│ localhost │ 9704 │ postgres      │                       14 │
└───────────┴──────┴───────────────┴──────────────────────────┘
(8 rows)

It is probably we cannot pass executor pool size to worker_split_copy

onderkalaci avatar Sep 20 '22 14:09 onderkalaci

related to https://github.com/citusdata/citus/issues/5289

onderkalaci avatar Sep 20 '22 14:09 onderkalaci

To be clear, this is basically #5289 but only for citus.max_adaptive_executor_pool_size to (hopefully) make it easier to implement.

JelteF avatar Sep 26 '22 09:09 JelteF