stable-diffusion-webui icon indicating copy to clipboard operation
stable-diffusion-webui copied to clipboard

RuntimeError: CUDA error: an illegal memory access was encountered

Open Kinomora opened this issue 2 years ago • 3 comments

Describe the bug Generating images has a slight chance of crashing the app due to "illegal memory access".

To Reproduce Steps to reproduce the behavior: Put in prompts, configure settings, press generate, crash happens randomly through the image generation process.

Settings used: Prompt: 19 tags Negative Prompt: 34 tags Sampling Steps 36 Sampling Method: Euler Width/Height: 512 Batch Count: 9 CFG Scale: 11 Stop At last layers of CLIP model: 2

Expected behavior Not crashing :)

Screenshots Crashlog/Stacktrace: https://hastebin.com/obinazobuw.txt TL;DR:

RuntimeError: CUDA error: an illegal memory access was encountered
CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1.

Desktop (please complete the following information):

  • OS: Windows 10 build 19044.2130
  • Browser: Chrome, 106.0.5249.119
  • Commit revision: cccc5a20fce4bde9a4299f8790366790735f1d05
  • Python 3.10.6
  • HW specs: Ryzen 5800X3D; 32GB of 3000MHz ram; software on NVMe SSD; RTX 2070 Super
  • Model: NovelAI hash [925997e9]

Additional context This also happened on my 3700X before upgrading; I currently have my bclk set to 100.5 (so 1000mhz = 1005mhz) on my system, no OC on my CPU, and my GPU is running at +105Mhz on the core and +1000mhz on the VRAM. Previously was running it at 1250 but was experiencing this issue much more frequently. Despite these OCs, I reset everything and still experience this crash.

Kinomora avatar Oct 17 '22 13:10 Kinomora

Similar issues: https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1766 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/1711 https://github.com/AUTOMATIC1111/stable-diffusion-webui/issues/234#issuecomment-1263157914

Kinomora avatar Oct 17 '22 13:10 Kinomora

This is almost certainly your hardware. Image generation in this fashion is more sensitive to memory corruption than most games, which will typically just glitch out for a couple of frames before returning to normal.

Just because you're not overclocking it doesn't mean it isn't having trouble. The problem can also be outside your GPU.

Baughn avatar Oct 17 '22 14:10 Baughn

I suppose I should provide my start script I'm using these arguments: COMMANDLINE_ARGS=--medvram --ckpt "C:\Users\Kinomora\Pictures\Stable Diffusion\stable-diffusion-webui\models\Stable-diffusion\nai.ckpt" --listen Additionally, I added "git pull" to the top of webui.bat

Kinomora avatar Oct 17 '22 14:10 Kinomora