rapidsai
[REVIEW][Java] One PinnedMemoryBuffer per CuVSResourcesImpl
While profiling cuvs-java, we found that allocating a PinnedMemoryBuffer for each host->device or device->host memory copy was unnecessary and wasteful.
This PR moves the allocation of a PinnedMemoryBuffer to CuVSResourcesImpl, so that the buffer can be cached and reused. Since CuVSResources are already meant to be per-thread, this is safe, as the PinnedMemoryBuffer will never be used concurrently.
In order to do it cleanly, we introduced two named ScopedAccess classes and a helper method that will always find its way to the internal MemorySegment used by native functions to access the buffer, without the need to expose it via the public interface.
This pull request requires additional validation before any workflows can run on NVIDIA's runners.
Pull request vetters can view their responsibilities here.
Contributors can view more details about this message here.
![copy-pr-bot[bot] avatar](https://avatars.githubusercontent.com/in/354886?v=4 "Published by a pub.dev verified publisher"){kind=small}
I've rebased this PR and resolved a conflict.
But it looks like we're going to need to fix the copyright headers on the changed files.
/ok to test d792fce
@benfred: This PR will likely continue to fail CI until the copyright headers are fixed for this change. Plus, there's another change pending (for handling cases where the row is super-large).
@mythrocks the copyright issue should be fixed with c4f0570 (but CI is still failing here on CagraMultiThreadStabilityIT.testQueryingUsingMultipleThreadsWithPrivateResources:62->testQueryingUsingMultipleThreads:162 MultiThreaded stablity test failed: null errors in the java unittests =(
@benfred @mythrocks I fixed the test to address the issue
I also addressed the issue with "too large rows" that won't fit in the buffer, falling back to a strategy that does direct copies (no buffering) in those cases (one for device -> host, the other for host -> device).
CI failures are a result of https://github.com/rapidsai/raft/pull/2813. Should be resolved by https://github.com/rapidsai/raft/pull/2881.