arrow
arrow copied to clipboard
ARROW-17330: [C#] Extend ArrowBuffer.BitmapBuilder to improve performance of array concatenation
Extend ArrowBuffer.BitmapBuilder with Append method overloaded with ReadOnlySpan
This allows to add validity bits to the builder more efficiently (especially for cases when initial validity bits are added to newly created empty builder). More over it makes BitmapBuilder API more consistent (for example ArrowBuffer.Builder<T> does have such method).
Currently adding new bits to existing bitmap is implemented in ArrayDataConcatenator, Code adds bit by bit in a cycle converting each to a boolean value:
for (int i = 0; i < length; i++) { builder.Append(span.IsEmpty || BitUtility.GetBit(span, i)); }
Thanks for opening a pull request!
If this is not a minor PR. Could you open an issue for this pull request on JIRA? https://issues.apache.org/jira/browse/ARROW
Opening JIRAs ahead of time contributes to the Openness of the Apache Arrow project.
Then could you also rename pull request title in the following format?
ARROW-${JIRA_ID}: [${COMPONENT}] ${SUMMARY}
or
MINOR: [${COMPONENT}] ${SUMMARY}
See also:
https://issues.apache.org/jira/browse/ARROW-17330
:warning: Ticket has not been started in JIRA, please click 'Start Progress'.
Thanks @asmirnov82 for the contribution. I didn't see this PR until now. I will make some time this week to review it.
- Closes: #32605
:warning: GitHub issue #32605 has been automatically assigned in GitHub to PR creator.
Benchmark runs are scheduled for baseline = 1de159d0f6763766c19b183dd309b8757723b43a and contender = cc20df18708886dd1de3889e1f99df9b59749212. cc20df18708886dd1de3889e1f99df9b59749212 is a master commit associated with this PR. Results will be available as each benchmark for each run completes.
Conbench compare runs links:
[Finished :arrow_down:0.0% :arrow_up:0.0%] ec2-t3-xlarge-us-east-2
[Failed] test-mac-arm
[Finished :arrow_down:0.77% :arrow_up:0.0%] ursa-i9-9960x
[Finished :arrow_down:2.77% :arrow_up:0.49%] ursa-thinkcentre-m75q
Buildkite builds:
[Finished] cc20df18
ec2-t3-xlarge-us-east-2
[Failed] cc20df18
test-mac-arm
[Finished] cc20df18
ursa-i9-9960x
[Finished] cc20df18
ursa-thinkcentre-m75q
[Finished] 1de159d0
ec2-t3-xlarge-us-east-2
[Failed] 1de159d0
test-mac-arm
[Finished] 1de159d0
ursa-i9-9960x
[Finished] 1de159d0
ursa-thinkcentre-m75q
Supported benchmarks:
ec2-t3-xlarge-us-east-2: Supported benchmark langs: Python, R. Runs only benchmarks with cloud = True
test-mac-arm: Supported benchmark langs: C++, Python, R
ursa-i9-9960x: Supported benchmark langs: Python, R, JavaScript
ursa-thinkcentre-m75q: Supported benchmark langs: C++, Java