DocMAX

And by the way there is no /sys/module/amdgpu/version. You have to correct the code.

> Ollama skipped the iGPU, because it has less than 1GB of VRAM. You have to configure VRAM allocation for the iGPU in BIOS to something like 8GB. Thanks i...

Totally agree!

> > But normal behaviour for the iGPU should be that it requests more VRAM if needed. > > Why do you think so? Where is it documented? Mine maxes...

Hmm, i see the model loaded into VRAM, but nothing happens... ``` llm_load_tensors: ggml ctx size = 0.22 MiB llm_load_tensors: offloading 32 repeating layers to GPU llm_load_tensors: offloading non-repeating layers...

Do i need another amdgpu module on the host than the one from the kernel (6.7.6)?

Hmm, tinyllama model does work with 5800U. The bigger ones stuck as i mentioned before. Edit: Codellama works too.

Very cool findings. Interesting you mention 96GB. I did a research and it seems thats the max. we can buy right now for SO-DIMMS. Wasn't aware it's called GTT. Let's...

OK, doesn't work with ollama. Wasn't aware that it doesn't use PyTorch right?

How does the env thing work? Like this? (Doesn't do anything btw) `LLAMA_HIP_UMA=1 HSA_OVERRIDE_GFX_VERSION=9.0.0 HCC_AMDGPU_TARGETS==gfx900 ollama start`