druid icon indicating copy to clipboard operation
druid copied to clipboard

MSQ: Use frames to back broadcast joinables.

Open gianm opened this issue 1 year ago • 2 comments

In BroadcastJoinSegmentMapFnProcessor, use FrameBasedInlineDataSource and FrameBasedIndexedTable to back broadcast joinables, rather than a regular InlineDataSource (which would use Java object arrays).

Reduces memory usage and eliminates a copy while building the joinable.

gianm avatar Aug 16 '24 17:08 gianm

AFAIR https://github.com/apache/druid/pull/14999 worked fine with the frame-based broadcast table and didn't have any performance degradation when compared with using columnar frames.

LakshSingla avatar Aug 20 '24 12:08 LakshSingla

@LakshSingla are you suggesting that in this patch I should use row-based frames rather than columnar?

gianm avatar Oct 10 '24 19:10 gianm

This pull request has been marked as stale due to 60 days of inactivity. It will be closed in 4 weeks if no further activity occurs. If you think that's incorrect or this pull request should instead be reviewed, please simply write any comment. Even if closed, you can still revive the PR at any time or discuss it on the [email protected] list. Thank you for your contributions.

github-actions[bot] avatar Dec 10 '24 00:12 github-actions[bot]

This pull request/issue has been closed due to lack of activity. If you think that is incorrect, or the pull request requires review, you can revive the PR at any time.

github-actions[bot] avatar Jan 08 '25 00:01 github-actions[bot]