Leon Sander comments

Repositories
Issues
Comments

Results 2 comments of


                                            Leon Sander

ollama can't run qwen:72b, error msg ""gpu VRAM usage didn't recover within timeout

@dhiltgen I have no problems on linux but get this error on windows. My application uses ollama as llm server, and many users work on windows and experience this error....

ollama can't run qwen:72b, error msg ""gpu VRAM usage didn't recover within timeout

@dhiltgen got it, thanks. Do you have an Idea why the loading/offloading on gpu takes that much time on windows? On linux llama3.1 is loaded in 10 seconds, but on...