CogVideo MAC下运行报错

System Info / 系統信息

Mac.已经安装了PyTorch, 运行失败报错

Information / 问题信息

[X] The official example scripts / 官方的示例脚本
[ ] My own modified scripts / 我自己修改的脚本和任务

Reproduction / 复现过程

(cogvideo) taozhiyu@Mac CogVideo % PYTORCH_ENABLE_MPS_FALLBACK=1 python inference/gradio_web_demo.py
Loading checkpoint shards: 100%|███████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 14.94it/s]
Loading pipeline components...: 100%|██████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00,  8.37it/s]
Traceback (most recent call last):
  File "/Users/taozhiyu/Downloads/CogVideo/inference/gradio_web_demo.py", line 25, in <module>
    pipe.enable_sequential_cpu_offload()
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 1160, in enable_sequential_cpu_offload
    cpu_offload(model, device, offload_buffers=offload_buffers)
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/big_modeling.py", line 204, in cpu_offload
    attach_align_device_hook(
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 502, in attach_align_device_hook
    add_hook_to_module(module, hook, append=True)
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 160, in add_hook_to_module
    module = hook.init_hook(module)
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 104, in init_hook
    module = hook.init_hook(module)
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 311, in init_hook
    set_module_tensor_to_device(
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 408, in set_module_tensor_to_device
    new_value = old_value.to(device)
  File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/cuda/__init__.py", line 305, in _lazy_init
    raise AssertionError("Torch not compiled with CUDA enabled")
AssertionError: Torch not compiled with CUDA enabled
(cogvideo) taozhiyu@Mac CogVideo % pip3 show torch                                                  
Name: torch
Version: 2.4.0
Summary: Tensors and Dynamic neural networks in Python with strong GPU acceleration
Home-page: https://pytorch.org/
Author: PyTorch Team
Author-email: [email protected]
License: BSD-3
Location: /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages
Requires: filelock, fsspec, jinja2, networkx, sympy, typing-extensions
Required-by: accelerate, deepspeed, SwissArmyTransformer, torchaudio, torchvision
(cogvideo) taozhiyu@Mac CogVideo %

Expected behavior / 期待表现

请问如何解决

Aug 31 '24 09:08 taozhiyuai

安装torch请使用mac的安装方式，mps，另外需要将模型启动换成mps

Aug 31 '24 17:08 zRzRzRzRzRzRzR

安装torch请使用mac的安装方式，mps，另外需要将模型启动换成mps

1.pip3 show torch 显示信息正确,已经安装正确

2.改为mps还是不能运行

Sep 01 '24 12:09 taozhiyuai

(cogvideo) taozhiyu@Mac inference % pip install torch torchvision torchaudio --extra-index-url https://download.pytorch.org/whl/cuXXX Looking in indexes: https://pypi.mirrors.ustc.edu.cn/simple/, https://pypi.tuna.tsinghua.edu.cn/simple, https://mirrors.aliyun.com/pypi/simple/, https://download.pytorch.org/whl/cuXXX Requirement already satisfied: torch in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (2.4.0) Requirement already satisfied: torchvision in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (0.19.0) Requirement already satisfied: torchaudio in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (2.4.0) Requirement already satisfied: filelock in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torch) (3.15.4) Requirement already satisfied: typing-extensions>=4.8.0 in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torch) (4.12.2) Requirement already satisfied: sympy in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torch) (1.13.2) Requirement already satisfied: networkx in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torch) (3.3) Requirement already satisfied: jinja2 in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torch) (3.1.4) Requirement already satisfied: fsspec in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torch) (2024.6.1) Requirement already satisfied: numpy in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torchvision) (1.26.0) Requirement already satisfied: pillow!=8.3.*,>=5.3.0 in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from torchvision) (9.5.0) Requirement already satisfied: MarkupSafe>=2.0 in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from jinja2->torch) (2.1.5) Requirement already satisfied: mpmath<1.4,>=1.1.0 in /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages (from sympy->torch) (1.3.0)

Sep 02 '24 00:09 taozhiyuai

改为MPS为什么会有cuda错误，或许你应该clone我们最新的代码，并将运行设备启动到mps上即 .to('mps')

Sep 02 '24 02:09 zRzRzRzRzRzRzR

改为MPS为什么会有cuda错误，或许你应该clone我们最新的代码，并将运行设备启动到mps上即 .to('mps')

感谢协助.

1.已经重新下载仓库 2.我按主页介绍的 # For macOS with Apple Silicon users, Intel not supported, this maybe 20x slower than RTX 4090 PYTORCH_ENABLE_MPS_FALLBACK=1 python gradio_web_demo.py

结果是: (cogvideo) taozhiyu@Mac inference % PYTORCH_ENABLE_MPS_FALLBACK=1 python gradio_web_demo.py Loading checkpoint shards: 100%|██████████████████| 2/2 [00:00<00:00, 12.77it/s] Loading pipeline components...: 100%|█████████████| 5/5 [00:00<00:00, 7.11it/s] Traceback (most recent call last): File "/Users/taozhiyu/Downloads/CogVideo/inference/gradio_web_demo.py", line 25, in <module> pipe.enable_sequential_cpu_offload() File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/diffusers/pipelines/pipeline_utils.py", line 1160, in enable_sequential_cpu_offload cpu_offload(model, device, offload_buffers=offload_buffers) File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/big_modeling.py", line 204, in cpu_offload attach_align_device_hook( File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 502, in attach_align_device_hook add_hook_to_module(module, hook, append=True) File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 160, in add_hook_to_module module = hook.init_hook(module) File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 104, in init_hook module = hook.init_hook(module) File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/hooks.py", line 311, in init_hook set_module_tensor_to_device( File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/accelerate/utils/modeling.py", line 408, in set_module_tensor_to_device new_value = old_value.to(device) File "/Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/site-packages/torch/cuda/__init__.py", line 305, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled 3.请问具体改哪里启动到mps?

Sep 02 '24 08:09 taozhiyuai

你需要吧 pipe.enable_model_cpu_offload() pipe.enable_sequential_cpu_offload() 这两行注释掉，然后确保你的mac有64G的内存（表测的是显存而不是内存，如果你使用的是内存推理，实际占用远远大于显存） pipe.to("mps")

Sep 03 '24 02:09 zRzRzRzRzRzRzR

`(cogvideo) taozhiyu@Mac inference % python gradio_web_demo.py Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 9.14it/s] Loading pipeline components...: 100%|████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 6.13it/s] Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). /AppleInternal/Library/BuildRoots/3bedef24-4170-11ef-a1db-e2437461156c/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:850: failed assertion [MPSNDArray initWithDevice:descriptor:isTextureBacked:] Error: total bytes of NDArray > 2**32' zsh: abort python gradio_web_demo.py (cogvideo) taozhiyu@Mac inference % /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

可以打开页面,选择下面的示例提示词点击产生视频的按钮后出现上述错误

Sep 03 '24 07:09 taozhiyuai

输入简单的提示词,点击enhanced prompt button, 没有任何反应,也没有任何报错.

@zRzRzRzRzRzRzR

Sep 03 '24 08:09 taozhiyuai

(cogvideo) taozhiyu@Mac inference % PYTORCH_ENABLE_MPS_FALLBACK=1 python gradio_web_demo.py Loading checkpoint shards: 100%|███████████████████████████████████████████| 2/2 [00:00<00:00, 4.64it/s] Loading pipeline components...: 100%|██████████████████████████████████████| 5/5 [00:01<00:00, 3.94it/s] Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). /AppleInternal/Library/BuildRoots/3bedef24-4170-11ef-a1db-e2437461156c/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:850: failed assertion `[MPSNDArray initWithDevice:descriptor:isTextureBacked:] Error: total bytes of NDArray > 2**32' zsh: abort PYTORCH_ENABLE_MPS_FALLBACK=1 python gradio_web_demo.py (cogvideo) taozhiyu@Mac inference % /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

Sep 09 '24 10:09 taozhiyuai

@taozhiyuai 你解决那个问题了吗？我也遇到了，没有找到任何解决措施

Sep 09 '24 12:09 alpttex19

@taozhiyuai 你解决那个问题了吗？我也遇到了，没有找到任何解决措施

没, @zRzRzRzRzRzRzR 没有回复我.

Sep 09 '24 13:09 taozhiyuai

以下是我最新近况.可以进入主界面.但是输入提示词,点击生成视频,就会出错.如下

`(cogvideo) taozhiyu@Mac cogvideo % cd inference (cogvideo) taozhiyu@Mac inference % python gradio_web_demo.py Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 12.41it/s] Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 6.74it/s] Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). /AppleInternal/Library/BuildRoots/3bedef24-4170-11ef-a1db-e2437461156c/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:850: failed assertion `[MPSNDArray initWithDevice:descriptor:isTextureBacked:] Error: total bytes of NDArray > 2**32' zsh: abort python gradio_web_demo.py (cogvideo) taozhiyu@Mac inference % /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

(cogvideo) taozhiyu@Mac inference % PYTORCH_ENABLE_MPS_FALLBACK=1 python gradio_web_demo.py Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 8.95it/s] Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 6.35it/s] Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). /AppleInternal/Library/BuildRoots/3bedef24-4170-11ef-a1db-e2437461156c/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:850: failed assertion `[MPSNDArray initWithDevice:descriptor:isTextureBacked:] Error: total bytes of NDArray > 2**32' zsh: abort PYTORCH_ENABLE_MPS_FALLBACK=1 python gradio_web_demo.py (cogvideo) taozhiyu@Mac inference % /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

`

Sep 09 '24 13:09 taozhiyuai

streamlit就会出现下面的问题

`(cogvideo) taozhiyu@Mac inference % python streamlit_web_demo.py 2024-09-09 21:37:46.911 WARNING streamlit.runtime.scriptrunner_utils.script_run_context: Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.911 WARNING streamlit.runtime.scriptrunner_utils.script_run_context: Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.916 Warning: to view this Streamlit app on a browser, run it with the following command:

streamlit run streamlit_web_demo.py [ARGUMENTS]

2024-09-09 21:37:46.916 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.916 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.916 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.916 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.916 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:46.916 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:47.421 Thread 'Thread-1': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:47.422 Thread 'Thread-1': missing ScriptRunContext! This warning can be ignored when running in bare mode. Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 2/2 [00:01<00:00, 1.65it/s] Loading pipeline components...: 100%|████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 5/5 [00:02<00:00, 1.72it/s] 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Session state does not function when running a script without streamlit run 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.204 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. 2024-09-09 21:37:52.205 Thread 'MainThread': missing ScriptRunContext! This warning can be ignored when running in bare mode. (cogvideo) taozhiyu@Mac inference % `

Sep 09 '24 13:09 taozhiyuai

这个问题我好像也暂时解决不了，目前没有mac设备能测试，如果有了，我们测一下反馈

Sep 10 '24 00:09 zRzRzRzRzRzRzR

I can't seem to solve this problem for the time being. I don't have a Mac device to test it. If we have one, we can test it and give feedback.

I can test! Just tell me what you need

Sep 20 '24 19:09 alexaaaaaander

Can it is running on Mac using with torch like to.("mps")

Sep 22 '24 06:09 zRzRzRzRzRzRzR

`(cogvideo) taozhiyu@Mac inference % python gradio_web_demo.py Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████| 2/2 [00:00<00:00, 9.14it/s] Loading pipeline components...: 100%|████████████████████████████████████████████████████████████| 5/5 [00:00<00:00, 6.13it/s] Running on local URL: http://127.0.0.1:7860

To create a public link, set share=True in launch(). /AppleInternal/Library/BuildRoots/3bedef24-4170-11ef-a1db-e2437461156c/Library/Caches/com.apple.xbs/Sources/MetalPerformanceShaders/MPSCore/Types/MPSNDArray.mm:850: failed assertion [MPSNDArray initWithDevice:descriptor:isTextureBacked:] Error: total bytes of NDArray > 2**32' zsh: abort python gradio_web_demo.py (cogvideo) taozhiyu@Mac inference % /Users/taozhiyu/miniconda3/envs/cogvideo/lib/python3.10/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked semaphore objects to clean up at shutdown warnings.warn('resource_tracker: There appear to be %d '

可以打开页面,选择下面的示例提示词点击产生视频的按钮后出现上述错误

@zRzRzRzRzRzRzR Can it is running on Mac using with torch like to.("mps")

你的这个问题我已经回答了,这就是错误信息.

Sep 23 '24 13:09 taozhiyuai

I know, but I have no way to verify it, because I don't have a high-performance Mac that can load this model, therefore, we need support from the community to conduct testing and modifications

Sep 24 '24 12:09 zRzRzRzRzRzRzR

I am having the same issue on M2 Mac with 96 GB RAM: To create a public link, set share=True in launch(). optimizing Traceback (most recent call last): File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/gradio/queueing.py", line 536, in process_events response = await route_utils.call_process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/gradio/route_utils.py", line 322, in call_process_api output = await app.get_blocks().process_api( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/gradio/blocks.py", line 1935, in process_api result = await self.call_function( ^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/gradio/blocks.py", line 1520, in call_function prediction = await anyio.to_thread.run_sync( # type: ignore ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/anyio/to_thread.py", line 56, in run_sync return await get_async_backend().run_sync_in_worker_thread( ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 2405, in run_sync_in_worker_thread return await future ^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/anyio/_backends/_asyncio.py", line 914, in run result = context.run(func, *args) ^^^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/gradio/utils.py", line 826, in wrapper response = f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/gradio/utils.py", line 826, in wrapper response = f(*args, **kwargs) ^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/cogstudio.py", line 727, in generate latents, seed = infer( ^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/cogstudio.py", line 217, in infer init(name, image_input, video_input, dtype, full_gpu) File "/Users/kc/CogVideo/inference/gradio_composite_demo/cogstudio.py", line 57, in init init_txt2vid(name, dtype_str, full_gpu) File "/Users/kc/CogVideo/inference/gradio_composite_demo/cogstudio.py", line 85, in init_txt2vid optimize(pipe, full_gpu) File "/Users/kc/CogVideo/inference/gradio_composite_demo/cogstudio.py", line 77, in optimize _pipe.enable_sequential_cpu_offload() File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/diffusers/pipelines/pipeline_utils.py", line 1133, in enable_sequential_cpu_offload cpu_offload(model, device, offload_buffers=offload_buffers) File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/accelerate/big_modeling.py", line 204, in cpu_offload attach_align_device_hook( File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/accelerate/hooks.py", line 512, in attach_align_device_hook attach_align_device_hook( File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/accelerate/hooks.py", line 503, in attach_align_device_hook add_hook_to_module(module, hook, append=True) File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/accelerate/hooks.py", line 161, in add_hook_to_module module = hook.init_hook(module) ^^^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/accelerate/hooks.py", line 312, in init_hook set_module_tensor_to_device( File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/accelerate/utils/modeling.py", line 408, in set_module_tensor_to_device new_value = old_value.to(device) ^^^^^^^^^^^^^^^^^^^^ File "/Users/kc/CogVideo/inference/gradio_composite_demo/env/lib/python3.12/site-packages/torch/cuda/init.py", line 305, in _lazy_init raise AssertionError("Torch not compiled with CUDA enabled") AssertionError: Torch not compiled with CUDA enabled

Oct 01 '24 15:10 kuntal-c

i have changed it to MPS instead of CUDA (i have MacBook Pro M2 16GB) pipe = CogVideoXPipeline.from_pretrained( "THUDM/CogVideoX-5b", torch_dtype=torch.bfloat16).to("mps")

after running that i am getting low memory error: RuntimeError: MPS backend out of memory (MPS allocated: 18.10 GB, other allocations: 384.00 KB, max allowed: 18.13 GB). Tried to allocate 80.00 MB on private pool. Use PYTORCH_MPS_HIGH_WATERMARK_RATIO=0.0 to disable upper limit for memory allocations (may cause system failure).

i hope this will help you guys to debug and solve this

after changing it to cpu it start working but it is very very slow