Niels Braczek comments

Repositories
Issues
Comments

Results 1 comments of


Niels Braczek

CUDA error 2: out of memory (for a 33 billion param model, but I have 39GB of VRAM available across 4 GPUs)

See https://github.com/jmorganca/ollama/issues/618#issuecomment-1737547046 The `num_gpu` parameter solved the problem for me. On my machine (only 12G), ollama loaded 43 layers and failed with the same error as above, but runs smooth...