Vladimir Sotnikov

Probably a related issue: https://github.com/huggingface/transformers/issues/23174

You need to provide `CUDA_DOCKER_ARCH` to make it work. E.g., `make LLAMA_CUBLAS=1 CUDA_DOCKER_ARCH=sm_87 LLAMA_CUDA_F16=1 -j 10` for Jetson Orin or `make LLAMA_CUBLAS=1 CUDA_DOCKER_ARCH=sm_72 LLAMA_CUDA_F16=1 -j 10` for Jetson Xavier.

@eternitybt are you using CUDA 10? If so, updating it should fix the problem (at least it's now working on my Xavier NX 16GB).

@eternitybt I haven't updated it per se, I just installed JetPack 5.1.2 from scratch, it includes CUDA 11. If you could afford wiping your Jetson, that's probably the easiest solution