datafusion-ballista
datafusion-ballista copied to clipboard
Make fetch shuffle partition data in parallel
Which issue does this PR close?
Closes #208.
Rationale for this change
What changes are included in this PR?
Are there any user-facing changes?
Hi @andygrove, @thinkharderdev, @avantgardnerio, could you help review this PR?
@yahoNanJing Could you please explain a little about the error handling case ? For example, if one of the parallel fetch failed, what happened to the others ? And what happened if there are multiple parallel fetch failures, will all the error info correctly propagated to the stream consumer ?
@mingmwang Just added one commit to abort fast when error occurs.