ILGPU icon indicating copy to clipboard operation
ILGPU copied to clipboard

How to Achieve Zero-Copy Between CPU and GPU

Open fangjiajiajia opened this issue 6 months ago • 1 comments

Question

Using the CopyFromCPU and CopyToCPU methods is quite time-consuming. Can shared memory be used to achieve zero-copy? tank you for help

Environment

  • ILGPU version: [e.g., 1.5.1]
  • .NET version: [e.g., .NET 8]
  • Operating system: [e.g., Windows 10]
  • Hardware (if GPU-related): [e.g., NVIDIA GeForce GTX 1080]

Additional context

No response

fangjiajiajia avatar Sep 09 '25 11:09 fangjiajiajia

Please note that "zero-copy" just means that the GPU has to read from Host Memory rather than Device Memory. However, the cost of reading from Host Memory is considerably more expensive. Depending on your access pattern, you are normally worse-off when using "zero-copy".

Also be aware that ILGPU is running in .NET, which is a garbage collected language, so you need to take precautions to pin memory, to prevent it moving.

ILGPU provides a wrapper on top of Cuda and OpenCL. Both of those frameworks provide some form of "zero-copy" access to RAM on the Host, but they are not well-supported by ILGPU. eg. you have to create your own subclasses, and handle ArrayView creation yourself.

For example, this sample code makes use of AllocateHostMemory to read memory from the Host.

MoFtZ avatar Sep 21 '25 02:09 MoFtZ