kvikio
kvikio copied to clipboard
rapidsai
Add Initial Java Support for GDS to KvikIO
This PR is intended to add initial support for Java binding to GDS as part of the KvikIO library. In this PR are the minimal set of bindings required to support synchronous read and write IO operations via GDS as well as a single example to demonstrate how the bindings can be used alongside other CUDA libraries, such as JCuda. Full support for the GDS CuFile API, including batch and asynchronous IO, has not yet been implemented and more sophisticated error/exception handling is not yet in place. There is a README located within kvikio/java detailing how this new functionality can be compiled and built locally, along with detailed instructions on how to run the included usage example.
This pull request requires additional validation before any workflows can run on NVIDIA's runners.
Pull request vetters can view their responsibilities here.
Contributors can view more details about this message here.
![copy-pr-bot[bot] avatar](https://avatars.githubusercontent.com/in/354886?v=4 "Published by a pub.dev verified publisher"){kind=small}
I will not likely finish the updates here for a couple weeks due to other priorities, so this should be moved to 24.10
Quick first pass of review -- mostly focused on packaging and CI. Give my suggestions a try and I'll comment
/ok to testto run your next commits on CI.
@bdice I believe I have updated the PR with all of your suggestions, except for the versioning question you had. Please let me know if you had further thoughts there or if it seems that I missed anything!
I forgot to add that the APIs to take a raw pointer is far from ideal. It might be nice to have a separate library that is compiled against jcuda and the code kvikio jar so that you can hide the scary part of pulling the native address out of a Pointer and provide a cleaner/safer API.
@madsbk or @bdice would one of you be able to try and run this through the build tests? I have updated it to try and make it more agnostic to the build location. It works on my machine, but need to know if there are more build errors remaining to be fixed for the CI/CD machines.
@aslobodaNV Looks like there are CI build failures. I'm not sure how to diagnose this.
Error: Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.0:compile (default-compile) on project cufile: Fatal error compiling: error: invalid target release: 21 -> [Help 1]
@aslobodaNV Great work. I left some comments but we should be able to get this merged after they are addressed.
@madsbk Do you want to target 24.12 or 25.02?
I have no strong opinion but it might make sense to add some more features before we release? @aslobodaNV do you want this to be part of the 24.12 release? Or should we have it in the nightly channel until 25.02?
@madsbk Do you want to target 24.12 or 25.02?
I have no strong opinion but it might make sense to add some more features before we release? @aslobodaNV do you want this to be part of the 24.12 release? Or should we have it in the nightly channel until 25.02?
I'm fine with this being held in the nightly channel till 25.02. I expect I will have some more time next quarter to work on some improvements here that would be good before the full release. These last two quarters have not given me much time to add the nice-to-haves.
@aslobodaNV Great work. I left some comments but we should be able to get this merged after they are addressed.
@bdice Addressed all comments except the rapids url change. Please confirm that it should differ from the one used in the cudf repo.