grigio

@arch-user-france1 But I could run "stable diffusion rocm" on my hardware loading the model on RAM, I think flux.1 schnell quantized should work too

With stable diffusion I could load the model in RAM (I've 48GB) and it worked, flux1-schnell-Q2_K.gguf 4.01 GB and the missing 90Mb i don't think it's related to that

I think the issue is a regression in the latest versions of pytorch / rocm https://github.com/huggingface/autotrain-advanced/issues/737 https://github.com/ROCm/ROCm/issues/3580

> Try set `HIP_VISIBLE_DEVICES` and `ROCR_VISIBLE_DEVICES` to 1 for your docker setup. Right now it only has 512MB memory, which is likely using the iGPU I want to use the...

Up, why is this issue closed ?

@crr0004 I tried your config but it crashes when I try to load the model, I don't know why cuda is mentioned even if it's a AMD card https://github.com/ggml-org/llama.cpp/issues/14370

deepseek-r1 do not support `tools` try with qwen2.5:32b or llama 70b

as a workaround wget works ``` oniux wget -O - http://2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion/index.html --2025-05-16 23:20:12-- http://2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion/index.html Resolving 2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion (2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion)... 10.242.61.222, fec0:ba08:d9a7:cdfa:8cec:af8b:7e97:eb3c Connecting to 2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion (2gzyxa5ihm7nsggfxnu52rck2vv4rvmdlkiu3zzui5du4xyclen53wid.onion)|10.242.61.222|:80... connected. HTTP request sent, awaiting response... 200...

have you had success to run [flux.1 GGUF](https://openart.ai/workflows/onion/flux-gguf-q8-12gb/X5HzyhrKjW2jqHVCTnvT) on AMD ? I still get ``` Error occurred when executing FluxSamplerParams+: HIP out of memory. Tried to allocate 64.00 MiB. GPU...

it isn't implemented, but PR are welcome