Broken support for CUDA versions < 11.1

[primenko-v]

trafficstars

This commit breaks the compatibility with older CUDA versions, presumably < 11.1.

The culprit is cudaHostRegisterReadOnly parameter that is used in ggml-cuda.cu, but was only introduced in CUDA 11.1, if this table is to be trusted.

The consequence is that the build fails on older CUDA devices. In my case it was NVIDIA Jetson Nano, which unfortunately is stuck on CUDA 10.2.

It also appears that the ggml_backend_cuda_register_host_buffer function, which relies on the problematic cudaHostRegisterReadOnly parameter, is used exclusively by Llama, and not by Whisper itself.

@ggerganov do you think there might be a simple workaround here?

For now, I am simply using an older whisper.cpp version, and it builds without problems.

[primenko-v]

Not sure if this is useful, but here is a list of cudaHostRegister* parameters that exist in CUDA 10.2 API:

#define cudaHostRegisterDefault  0x00  /**< Default host memory registration flag */
#define cudaHostRegisterPortable 0x01  /**< Pinned memory accessible by all CUDA contexts */
#define cudaHostRegisterMapped   0x02  /**< Map registered memory into device space */
#define cudaHostRegisterIoMemory 0x04  /**< Memory-mapped I/O space */

[JuntongLiu96]

Same issue here

[slaren]

You can enclose the incompatible code in a #if CUDART_VERSION >= 11100 block or similar, and return false for older versions. A PR to do this would be welcome.