Ken, Wang
Ken, Wang
I can work on this improvement.
Other improvement I can think of is if the shuffle data was colocated on the same host with the shuffle reader, we should allow the reader to read from disk...
@metesynnada @andygrove Sorry, I do not get a chance to look into this. Regarding the issue, I'm not sure we would like to fix it in this way. In my...
Let the CompletedTask return the partition stats is quite heavy. Imaging we have 1000 map tasks and 1000 reduce tasks(partition = 1000), the stats will become 1M.
I can work on this.
Ideally we can implement the bubble execution model which mix the pipeline and batch execution model in one engine. Based on the complexity of the query DAG, some parts can...
There are huge gaps in DataFusion's physical planning phase, I think we need to put more effort to address those gaps first. Only with efficient and sophisticated physical planner, we...
> I also have some thoughts about a unified execution engine, welcome to take a look and comment: > > https://www.notion.so/liurenjie1024/A-Cloud-Native-Universal-Execution-Engine-7903dd9eeea143c48049631a2d1cb845 > > cc @andygrove @mingmwang How can I get...
Is it still an issue?