thu-ml
Required configuration
Hello, I would like to inquire about the hardware requirements to run this work. My running environment is A40 48G, but the following error occurs when I run the sample:
RuntimeError: CUDA error: invalid configuration argument CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.
My environment is A100 80G and i have the same problem, for the virtual environment, i just directly use the one from fatezero, when it finished 300 steps, it reports the same error as yours, and i cannot use the environment.yml to get the ofiicial running environment, it seems that there is something wrong with the xformers, i wonder if you have the same problem? Now i am trying to use RTX4090 to see if it the problem of the GPU.
Finally, i partly solved the problem, i found that it is the xformers that causes the problem. If you set the config enable_xformers_memory_efficient_attention: False, and you don't have any other problems, you can solve the current problem, and get the video you want, but you should have enough GPU memory, for the example command, it takes 30572MiB, which is much larger than the Xformers version. I am still trying to solve this Xformers problem, i think it has to do with the CUDA verison and the xformers version,if you have any findings, please let me know, thanks
Have someone solve this error?
I successfully run the demo by installing a former version of xformers. In my case, I installed xformers==0.0.19, hope it's helpful!
Finally, i partly solved the problem, i found that it is the xformers that causes the problem. If you set the config enable_xformers_memory_efficient_attention: False, and you don't have any other problems, you can solve the current problem, and get the video you want, but you should have enough GPU memory, for the example command, it takes 30572MiB, which is much larger than the Xformers version. I am still trying to solve this Xformers problem, i think it has to do with the CUDA verison and the xformers version,if you have any findings, please let me know, thanks
How did you "set enable_xformers_memory_efficient_attention: False"?Because when i change the 55th code line in train.py from"enable_xformers_memory_efficient_attention: bool = True,"to "enable_xformers_memory_efficient_attention: bool = False,"I got another error as follows:
RuntimeError: No CUDA GPUs are available
Finally, i partly solved the problem, i found that it is the xformers that causes the problem. If you set the config enable_xformers_memory_efficient_attention: False, and you don't have any other problems, you can solve the current problem, and get the video you want, but you should have enough GPU memory, for the example command, it takes 30572MiB, which is much larger than the Xformers version. I am still trying to solve this Xformers problem, i think it has to do with the CUDA verison and the xformers version,if you have any findings, please let me know, thanks
How did you "set enable_xformers_memory_efficient_attention: False"?Because when i change the 55th code line in train.py from"enable_xformers_memory_efficient_attention: bool = True,"to "enable_xformers_memory_efficient_attention: bool = False,"I got another error as follows: RuntimeError: No CUDA GPUs are available
I confirm my cuda is available after run checkcuda.py.
<img width="450" alt="image" src="https://github.com/thu-
ml/controlvideo/assets/117589014/b2a059f9-7558-4809-ae67-2f3dd6fe46ad">