pg_duckdb
pg_duckdb copied to clipboard
Parallel Scan with multiple processes
- We can now start multiple worker process that can read relation blocks and write buffer to shared memory to be consumed by duckdb reader threads. Problem can be viewed as variant of multi producer / multi consumer where producer are responsible for releasing buffer after they are read.
@mkaruza what's the deal with this? I guess it's not critical for 0.1.0. Given the amount of merge conflicts it has now, I'm think it probably makes the most sense to simply close this and maybe create an issue for it instead.
@JelteF agree not critical, but could be approach to get better performances of heap table scans (specifically to fetch table pages in parallel; currently there is bottleneck there as only one thread in process can request page fetching).
@mkaruza do you want to close this one in favor of #477
Closing