AUTOMATIC1111
support cuda malloc
imported from comfy: https://github.com/comfyanonymous/ComfyUI/blob/f1d6cef71c70719cc3ed45a2455a4e5ac910cd5e/cuda_malloc.py
original commits:
- https://github.com/comfyanonymous/ComfyUI/commit/799c08a: Auto disable cuda malloc on some GPUs on windows.
- https://github.com/comfyanonymous/ComfyUI/commit/D39c58b: Disable cuda malloc on GTX 750 Ti.
- https://github.com/comfyanonymous/ComfyUI/commit/85a8900: Disable cuda malloc on regular GTX 960.
- https://github.com/comfyanonymous/ComfyUI/commit/30de083: Disable cuda malloc on all the 9xx series.
- https://github.com/comfyanonymous/ComfyUI/commit/7c0a5a3: Disable cuda malloc on a bunch of quadro cards.
- https://github.com/comfyanonymous/ComfyUI/commit/5a90d3c: GeForce MX110 + MX130 are maxwell.
- https://github.com/comfyanonymous/ComfyUI/commit/fc71cf6: Add some 800M gpus to cuda malloc blacklist.
- https://github.com/comfyanonymous/ComfyUI/commit/861fd58: Add a warning if a card that doesn't support cuda malloc has it enabled.
- https://github.com/comfyanonymous/ComfyUI/commit/192ca06: Add some more cards to the cuda malloc blacklist.
- https://github.com/comfyanonymous/ComfyUI/commit/caddef8: Auto disable cuda malloc on unsupported GPUs on Linux.
- https://github.com/comfyanonymous/ComfyUI/commit/2f93b91: Add Tesla GPUs to cuda malloc blacklist.
Description
detect supported GPUs and add PYTORCH_CUDA_ALLOC_CONF=backend:cudaMallocAsync automatically.
- [x] support
cuda_malloc - [x] add
-cuda-malloc,--disable-cuda-malloccmd args
Checklist:
- [x] I have read contributing wiki page
- [x] I have performed a self-review of my own code
- [x] My code follows the style guidelines
- [x] My code passes tests
@wkpark thank you so much
can you explain what cuda malloc exactly does and brings performance or memory optimization?
@wkpark thank you so much
can you explain what cuda malloc exactly does and brings performance or memory optimization?
Please refer to the following articles: https://iamholumeedey007.medium.com/memory-management-using-pytorch-cuda-alloc-conf-dabe7adec130
and https://github.com/comfyanonymous/ComfyUI/commit/50bf66e5c44fe3637f29999034c10a0c083c7600
@wkpark ty so much
why comfyui made it disabled by default do you know? we are all moved to torch 2.4
@wkpark ty so much
why comfyui made it disabled by default do you know? we are all moved to torch 2.4
that's not the latest change. already reverted two month ago. https://github.com/comfyanonymous/ComfyUI/commits/master/cuda_malloc.py
Just wanted to let devs know. I've been using this PR with the startup tag active for three months now without issue. Running a RTX2070 laptop. https://github.com/aleph23/auto1111/pull/7
