Hui Chen

Lite image

2

Is there a lite version of this image, ie no desktop, for server usage?

[User] How to specify which cuda device to use programmably

2

Say I have four Nvidia cards and I want to run four models on each of the card in one program. The SDK doesn't provide parameter to specify which cuda...

Cuda runtime error and slow eval

3

Using [a09f919](https://github.com/ggerganov/llama.cpp/releases/tag/master-a09f919) and compiled with ``` make clean && LLAMA_CUBLAS=1 LLAMA_CUDA_DMMV_X=64 LLAMA_CUDA_DMMV_Y=2 make -j ``` Running with command, on 4x A40-48G ``` ./main -m ggml-vic13b-q5_1.bin -ngl 1000 -p "the meaning...