cccl icon indicating copy to clipboard operation
cccl copied to clipboard

Implement `cudax::cufile`

Open davebayer opened this issue 2 months ago • 13 comments

This PR implements wrappers of CUfileHandle_t and related APIs.

It introduces:

  1. cudax::cufile type which handles opening/closing the native file handle and registering/deregistering of the cuFile file handle. It owns both of these resources.
  2. cudax::cufile_ref type that is a lightweight non-owning wrapper for the cuFile file handle. It doesn't own the resources (with exception below).
  3. cuda::cufile_driver.[de]register_native_handle(...) -> cuda::cufile_ref that can be used for manual registration of the native file handle. It has basically the same semantics as new/delete.

So, right now these are the scenarios are covered:

  1. the user has some legacy code from where he gets the CUfileHandle_t -> he can use cuda::cufile_ref with our APIs.
  2. the user has some code that gives him the native file handle, but he doesn't own the native file handle -> he can use cuda::cufile_driver.register_native_handle(...).
  3. the user wants us to open the native file handle for him as well -> he can use cuda::cufile

We might introduce a third option: cuda::scoped_cufile or something similar, that would be constructible from cuda::cufile_ref and would deregister the handle when going out of scope, but I would wait with it.

davebayer avatar Oct 03 '25 09:10 davebayer

Auto-sync is disabled for draft pull requests in this repository. Workflows must be run manually.

Contributors can view more details about this message here.

copy-pr-bot[bot] avatar Oct 03 '25 09:10 copy-pr-bot[bot]

😬 CI Workflow Results

🟥 Finished in 26m 23s: Pass: 92%/42 | Total: 3h 27m | Max: 20m 41s | Hits: 99%/19806

See results here.

github-actions[bot] avatar Oct 16 '25 14:10 github-actions[bot]

Fair enough, those are valid points. I was unsure we needed it but as you said the easier of use and known methods are totally valid and I think it's good to have!

Enviado desde Outlook para Androidhttps://aka.ms/AAb9ysg


From: David Bayer @.> Sent: Thursday, October 16, 2025 9:15:39 PM To: NVIDIA/cccl @.> Cc: Javier Vera @.>; Comment @.> Subject: Re: [NVIDIA/cccl] Implement cudax::cufile (PR #6122)

@davebayer commented on this pull request.


In cudax/include/cuda/experimental/__cufile/cufile.cuhhttps://github.com/NVIDIA/cccl/pull/6122#discussion_r2437195477:

  • _CCCL_HOST_API void open(const char* __filename, cufile_open_mode __open_mode)
  • {
  • if (is_open())
  • {
  •  ::cuda::std::__throw_runtime_error("File is already opened.");
    
  • }
  • _native_handle = __open_file(__filename, __make_oflags(__open_mode));
  • try
  • {
  •  __cufile_handle_ = __register_cufile_handle(__native_handle_);
    
  • }
  • catch (...)
  • {
  •  __close_file(::cuda::std::exchange(__native_handle_, __invalid_native_handle));
    
  •  throw;
    
  • }
  • }

That's true, but I'd like to provide tools users are used to have. Consider this:

cuda::cufile file; // ... file = cuda::cufile{filename, open_mode}; // open the file // ... file = cuda::cufile{}; // close the file

You have a situation like this, when you want to have the object in not opened state. Then, to open the file, you must create another object and move assign it to the original object. And if you want to close the file with detection whether it was closed or not, you must (again) move assign a new default constructed object.

— Reply to this email directly, view it on GitHubhttps://github.com/NVIDIA/cccl/pull/6122#discussion_r2437195477, or unsubscribehttps://github.com/notifications/unsubscribe-auth/BSUF2R623U6KWV3NPWPNYAL3X7VFXAVCNFSM6AAAAACIGCYCK2VHI2DSMVQWIX3LMV43YUDVNRWFEZLROVSXG5CSMV3GSZLXHMZTGNBWGYZDKOJZGA. You are receiving this because you commented.Message ID: @.***>

jvera-nvidia avatar Oct 16 '25 20:10 jvera-nvidia

😬 CI Workflow Results

🟥 Finished in 20m 03s: Pass: 92%/42 | Total: 2h 30m | Max: 16m 44s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 16 '25 21:10 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 24m 25s: Pass: 92%/42 | Total: 2h 48m | Max: 19m 29s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 17 '25 07:10 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 34m 14s: Pass: 92%/42 | Total: 2h 57m | Max: 29m 25s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 17 '25 08:10 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 37m 21s: Pass: 92%/42 | Total: 5h 25m | Max: 25m 59s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 20 '25 06:10 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 16m 11s: Pass: 92%/42 | Total: 2h 30m | Max: 14m 58s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 20 '25 19:10 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 31m 04s: Pass: 92%/42 | Total: 3h 10m | Max: 26m 08s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 20 '25 19:10 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 26m 51s: Pass: 92%/42 | Total: 3h 10m | Max: 23m 12s | Hits: 99%/19828

See results here.

github-actions[bot] avatar Oct 20 '25 20:10 github-actions[bot]

Waiting for https://github.com/nv-gha-runners/roadmap/issues/318

davebayer avatar Oct 23 '25 08:10 davebayer

Waiting for https://github.com/nv-gha-runners/roadmap/issues/318

As discussed internally, we will just build the tests for now, we will run those tests once the support is added. I tested it locally with CUDA 12.9 and 13.0 and both compiled & ran fine.

davebayer avatar Nov 14 '25 09:11 davebayer

🥳 CI Workflow Results

🟩 Finished in 4h 18m: Pass: 100%/261 | Total: 7d 15h | Max: 4h 17m | Hits: 85%/379106

See results here.

github-actions[bot] avatar Nov 14 '25 13:11 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 2h 21m: Pass: 97%/261 | Total: 3d 14h | Max: 1h 41m | Hits: 99%/370640

See results here.

github-actions[bot] avatar Nov 17 '25 21:11 github-actions[bot]

😬 CI Workflow Results

🟥 Finished in 10h 43m: Pass: 99%/261 | Total: 3d 16h | Max: 1h 41m | Hits: 99%/377801

See results here.

github-actions[bot] avatar Nov 18 '25 06:11 github-actions[bot]

🥳 CI Workflow Results

🟩 Finished in 12h 12m: Pass: 100%/261 | Total: 3d 16h | Max: 1h 41m | Hits: 99%/379106

See results here.

github-actions[bot] avatar Nov 18 '25 07:11 github-actions[bot]