spark icon indicating copy to clipboard operation
spark copied to clipboard

Published 20 hours ago • verified publisher icon apache

[SPARK-40487][SQL] Make defaultJoin in BroadcastNestedLoopJoinExec running in parallel

Open xingchaozh opened this issue 2 years ago • 3 comments

What changes were proposed in this pull request?

Currently, the defaultJoin method in BroadcastNestedLoopJoinExec collects notMatchedBroadcastRows firstly, then collects matchedStreamRows. The two steps could run in parallel instead of serial.

Why are the changes needed?

Make defaultJoin in BroadcastNestedLoopJoinExec running in parallel

Does this PR introduce any user-facing change?

NO

How was this patch tested?

UT

xingchaozh avatar Sep 19 '22 10:09 xingchaozh

Can one of the admins verify this patch?

AmplabJenkins avatar Sep 19 '22 17:09 AmplabJenkins

@cloud-fan @c21

wangyum avatar Sep 20 '22 06:09 wangyum

makes sense, also cc @maryannxue @somani

cloud-fan avatar Sep 20 '22 07:09 cloud-fan

Merged to master.

wangyum avatar Sep 22 '22 03:09 wangyum