LightX2V
LightX2V copied to clipboard
Published
4 days ago
•
ModelTC
ModelTC
[Bug] docker run wan2.2+sageattention error
Description
I use docker registry.cn-hangzhou.aliyuncs.com/yongyang/lightx2v:25080601-cu124-SageSm89 Go run wan2.2+sageattention+lightx lora
Steps to Reproduce
- comfyui run wan2.2
Expected Result
model normal inference
Actual Result
Using 1053 LoRA weight patches for WanVideo model
Could not set recompile_limit: torch._dynamo.config.recompile_limit does not exist
/data/code/haobang.geng/ComfyUI/custom_nodes/ComfyUI-WanVideoWrapper/utils.py:495: FutureWarning: Accessing config attribute `vace_layers` directly via 'WanModel' object attribute is deprecated. Please access 'vace_layers' over 'WanModel's config object instead, e.g. 'unet.config.vace_layers'.
if transformer.vace_layers is not None:
timesteps: tensor([999, 975, 941, 888, 799, 615], device='cuda:0')
sigmas: tensor([1.0000, 0.9756, 0.9412, 0.8889, 0.8000, 0.6153, 0.0000])
Sampling until step 3, timestep: 941
timesteps: tensor([999, 975, 941], device='cuda:0')
image_cond shape: torch.Size([20, 11, 60, 104])
/data/code/haobang.geng/ComfyUI/custom_nodes/ComfyUI-WanVideoWrapper/nodes.py:1805: FutureWarning: Accessing config attribute `vace_layers` directly via 'WanModel' object attribute is deprecated. Please access 'vace_layers' over 'WanModel's config object instead, e.g. 'unet.config.vace_layers'.
if transformer.vace_layers is not None:
Swapping 20 transformer blocks
Initializing block swap: 0%| | 0/40 [00:00<?, ?it/s]FETCH ComfyRegistry Data: 10/94
Initializing block swap: 92%|███████████████████████████████████████████████████████████████████████▏ | 37/40 [00:13<00:02, 1.22it/s]FETCH ComfyRegistry Data: 15/94
Initializing block swap: 100%|█████████████████████████████████████████████████████████████████████████████| 40/40 [00:18<00:00, 2.12it/s]
/data/code/haobang.geng/ComfyUI/custom_nodes/ComfyUI-WanVideoWrapper/wanvideo/modules/model.py:1270: FutureWarning: Accessing config attribute `vace_layers` directly via 'WanModel' object attribute is deprecated. Please access 'vace_layers' over 'WanModel's config object instead, e.g. 'unet.config.vace_layers'.
if vace_blocks_to_swap > 0 and self.vace_layers is not None:
----------------------
Block swap memory summary:
Transformer blocks on cpu: 6704.63MB
Transformer blocks on cuda:0: 6704.63MB
Total memory used by transformer blocks: 13409.26MB
Non-blocking memory transfer: False
----------------------
Seq len: 17160
Sampling 41 frames at 832x480 with 6 steps
0%| | 0/3 [00:00<?, ?it/s]FETCH ComfyRegistry Data: 20/94
terminate called after throwing an instance of 'c10::Error'
what(): CUDA error: unspecified launch failure
CUDA kernel errors might be asynchronously reported at some other API call, so the stacktrace below might be incorrect.
For debugging consider passing CUDA_LAUNCH_BLOCKING=1
Compile with `TORCH_USE_CUDA_DSA` to enable device-side assertions.
Exception raised from c10_cuda_check_implementation at ../c10/cuda/CUDAException.cpp:43 (most recent call first):
frame #0: c10::Error::Error(c10::SourceLocation, std::string) + 0x96 (0x7fd9ca2b9446 in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #1: c10::detail::torchCheckFail(char const*, char const*, unsigned int, std::string const&) + 0x64 (0x7fd9ca2636e4 in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #2: c10::cuda::c10_cuda_check_implementation(int, char const*, char const*, int, bool) + 0x118 (0x7fd9ca3a5a18 in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10_cuda.so)
frame #3: <unknown function> + 0x5e460 (0x7fd9ca3ab460 in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10_cuda.so)
frame #4: <unknown function> + 0x60184 (0x7fd9ca3ad184 in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10_cuda.so)
frame #5: <unknown function> + 0x5fcdd0 (0x7fd9c918bdd0 in /opt/conda/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
frame #6: <unknown function> + 0x6f69f (0x7fd9ca29a69f in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #7: c10::TensorImpl::~TensorImpl() + 0x21b (0x7fd9ca29337b in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #8: c10::TensorImpl::~TensorImpl() + 0x9 (0x7fd9ca293529 in /opt/conda/lib/python3.11/site-packages/torch/lib/libc10.so)
frame #9: at::autocast::clear_cache() + 0x100 (0x7fd9b5852990 in /opt/conda/lib/python3.11/site-packages/torch/lib/libtorch_cpu.so)
frame #10: <unknown function> + 0x84a552 (0x7fd9c93d9552 in /opt/conda/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
<omitting python frames>
frame #14: <unknown function> + 0x8ee784 (0x7fd9c947d784 in /opt/conda/lib/python3.11/site-packages/torch/lib/libtorch_python.so)
frame #32: <unknown function> + 0x76a6 (0x7fd9cc02d6a6 in /opt/conda/lib/python3.11/lib-dynload/_asyncio.cpython-311-x86_64-linux-gnu.so)
frame #43: <unknown function> + 0x94ac3 (0x7fda6e11cac3 in /usr/lib/x86_64-linux-gnu/libc.so.6)
frame #44: clone + 0x44 (0x7fda6e1ada04 in /usr/lib/x86_64-linux-gnu/libc.so.6)
Aborted (core dumped)
Environment Information
Python environment
pip list
Package Version Editable project location
--------------------------------- ------------- -------------------------
absl-py 2.3.1
accelerate 1.9.0
addict 2.4.0
aggdraw 1.3.19
aiofiles 24.1.0
aiohappyeyeballs 2.6.1
aiohttp 3.12.15
aiohttp-retry 2.8.3
aiohttp_socks 0.10.1
aioice 0.10.1
aiortc 1.9.0
aiosignal 1.4.0
aiosqlite 0.21.0
albucore 0.0.16
albumentations 1.4.4
alembic 1.16.4
annotated-types 0.7.0
anthropic 0.34.2
antlr4-python3-runtime 4.9.3
anyio 4.10.0
APScheduler 3.11.0
archspec 0.2.3
astor 0.8.1
asttokens 2.4.1
astunparse 1.6.3
asyncua 1.1.6
attrs 24.2.0
autocommand 2.2.2
av 15.0.0
backoff 2.2.1
backports.tarfile 1.2.0
beautifulsoup4 4.12.3
bitsandbytes 0.47.0
black 25.1.0
blake3 1.0.5
blend_modes 2.2.0
blind-watermark 0.4.4
boltons 24.0.0
boto3 1.35.60
botocore 1.35.99
Brotli 1.1.0
cachetools 5.5.2
cbor2 5.6.5
certifi 2024.8.30
cffi 1.17.1
cfgv 3.4.0
chardet 5.2.0
charset-normalizer 3.4.0
click 8.1.7
cloudpickle 3.1.1
cmake 4.0.3
color-matcher 0.6.0
colorama 0.4.6
coloredlogs 15.0.1
colour-science 0.4.6
comfyui-embedded-docs 0.2.4
comfyui_frontend_package 1.23.4
comfyui_workflow_templates 0.1.44
commonmark 0.9.1
compressed-tensors 0.10.2
conda 24.9.2
conda-build 24.9.0
conda_index 0.5.0
conda-libmamba-solver 24.9.0
conda-package-handling 2.4.0
conda_package_streaming 0.11.0
contourpy 1.3.3
cryptography 45.0.6
cstr 0.1.0
cupy-cuda12x 13.5.1
cycler 0.12.1
Cython 3.0.12
dataclasses-json 0.6.7
ddt 1.7.2
decorator 5.1.1
deepdiff 8.6.0
defusedxml 0.7.1
depyf 0.19.0
diffusers 0.34.0
dill 0.3.8
diskcache 5.6.3
distlib 0.4.0
distro 1.9.0
dnspython 2.7.0
docker 7.1.0
docopt 0.6.2
docutils 0.22
easydict 1.13
einops 0.8.1
email_validator 2.2.0
exceptiongroup 1.2.2
executing 2.1.0
expecttest 0.2.1
fairscale 0.4.0
fastapi 0.110.3
fastapi-cli 0.0.8
fastapi-cloud-cli 0.1.5
fastrlock 0.8.3
ffmpy 0.3.0
filelock 3.16.1
flash_attn 2.8.2
flash_attn 2.8.2
flash_attn_3 3.0.0b1
flash_attn_3 3.0.0b1
flatbuffers 25.2.10
flet 0.28.3
fonttools 4.59.1
frozendict 2.4.6
frozenlist 1.7.0
fsspec 2024.10.0
ftfy 6.3.1
future 1.0.0
gdown 5.2.0
gguf 0.17.1
gitdb 4.0.12
GitPython 3.1.45
google-ai-generativelanguage 0.6.15
google-api-core 2.25.1
google-api-python-client 2.179.0
google-auth 2.40.3
google-auth-httplib2 0.2.0
google-crc32c 1.7.1
google-generativeai 0.8.5
googleapis-common-protos 1.70.0
greenlet 3.2.4
grpcio 1.74.0
grpcio-status 1.62.3
h11 0.16.0
h2 4.1.0
hf-xet 1.1.7
hpack 4.0.0
httpcore 1.0.9
httplib2 0.22.0
httptools 0.6.4
httpx 0.27.2
huggingface-hub 0.34.3
humanfriendly 10.0
hydra-core 1.3.2
hyperframe 6.0.1
hypothesis 6.115.5
identify 2.6.12
idna 3.10
ifaddr 0.2.0
imageio 2.37.0
imageio-ffmpeg 0.6.0
img2texture 1.0.6
importlib_metadata 8.7.0
importlib_resources 6.4.5
inference-cli 0.45.1
inference-gpu 0.45.1
inflect 7.3.1
iniconfig 2.1.0
interegular 0.3.3
iopath 0.1.10
ipython 8.29.0
isort 6.0.1
jaraco.collections 5.1.0
jaraco.context 5.3.0
jaraco.functools 4.0.1
jaraco.text 3.12.1
jax 0.7.0
jaxlib 0.7.0
jedi 0.19.1
Jinja2 3.1.6
jiter 0.10.0
jmespath 1.0.1
joblib 1.5.1
jsonpatch 1.33
jsonpointer 3.0.0
jsonschema 4.23.0
jsonschema-specifications 2024.10.1
kiwisolver 1.4.9
kornia 0.8.1
kornia_rs 0.1.9
lark 1.2.2
lazy_loader 0.4
libarchive-c 5.1
libmambapy 1.5.10
lief 0.14.1
lintrunner 0.12.5
llguidance 0.7.30
llvmlite 0.44.0
lm-format-enforcer 0.10.12
loguru 0.7.3
Mako 1.3.10
mamba 1.5.10
markdown-it-py 3.0.0
MarkupSafe 3.0.2
marshmallow 3.26.1
matplotlib 3.10.5
matplotlib-inline 0.1.7
matrix-nio 0.25.2
mdurl 0.1.2
mediapipe 0.10.21
menuinst 2.1.2
mistral_common 1.8.3
ml_dtypes 0.5.3
more-itertools 10.5.0
mpmath 1.3.0
msgpack 1.1.1
msgspec 0.19.0
mss 10.1.0
multidict 6.6.3
mypy_extensions 1.1.0
networkx 3.4
ninja 1.11.1.4
nodeenv 1.9.1
numba 0.61.2
numpy 1.26.4
nvidia-cublas-cu12 12.4.5.8
nvidia-cuda-cupti-cu12 12.4.127
nvidia-cuda-nvrtc-cu12 12.4.127
nvidia-cuda-runtime-cu12 12.4.127
nvidia-cudnn-cu12 9.1.0.70
nvidia-cufft-cu12 11.2.1.3
nvidia-curand-cu12 10.3.5.147
nvidia-cusolver-cu12 11.6.1.9
nvidia-cusparse-cu12 12.3.1.170
nvidia-ml-py 12.575.51
nvidia-nccl-cu12 2.21.5
nvidia-nvjitlink-cu12 12.4.127
nvidia-nvtx-cu12 12.4.127
oauthlib 3.3.1
omegaconf 2.3.0
onnxruntime 1.22.1
onnxruntime-gpu 1.19.2
openai 1.99.1
openai-harmony 0.0.3
opencv-contrib-python 4.10.0.84
opencv-python 4.10.0.84
opencv-python-headless 4.11.0.86
opt_einsum 3.4.0
optree 0.13.0
orderly-set 5.5.0
outlines_core 0.2.10
packaging 24.2
paho-mqtt 1.6.1
pandas 2.2.3
parso 0.8.4
partial-json-parser 0.2.1.1.post6
pathspec 0.12.1
peft 0.17.0
pexpect 4.9.0
pickleshare 0.7.5
piexif 1.1.3
pilgram 1.2.1
pillow 10.4.0
pip 24.2
pkginfo 1.11.2
pkgutil_resolve_name 1.3.10
platformdirs 4.3.6
pluggy 1.5.0
pooch 1.8.2
portalocker 3.2.0
pre_commit 4.2.0
prometheus_client 0.22.1
prometheus-fastapi-instrumentator 6.0.0
prompt_toolkit 3.0.48
propcache 0.3.2
proto-plus 1.26.1
protobuf 4.25.8
psd-tools 1.10.9
psutil 6.1.0
ptyprocess 0.7.0
pure_eval 0.2.3
py-cpuinfo 9.0.0
pyasn1 0.6.1
pyasn1_modules 0.4.2
pybase64 1.0.2
pycosat 0.6.6
pycountry 24.6.1
pycparser 2.22
pycryptodome 3.23.0
pydantic 2.6.4
pydantic_core 2.16.3
pydantic-extra-types 2.10.5
pydantic-settings 2.2.1
pydot 2.0.0
pyee 13.0.0
PyGithub 2.7.0
Pygments 2.18.0
PyJWT 2.10.1
pylibsrtp 0.12.0
pylogix 1.0.5
pyloudnorm 0.1.1
PyMatting 1.1.14
pymodbus 3.8.3
PyNaCl 1.5.0
pynvml 12.0.0
pyOpenSSL 25.1.0
pyparsing 3.2.3
PySocks 1.7.1
pytest 8.4.1
python-dateutil 2.9.0.post0
python-dotenv 1.0.1
python-etcd 0.4.5
python-json-logger 3.3.0
python-multipart 0.0.20
python-socks 2.7.2
pytz 2024.2
PyWavelets 1.9.0
PyYAML 6.0.2
pyzbar 0.1.9
pyzmq 27.0.1
qrcode 8.2
qtorch 0.3.0
ray 2.48.0
redis 5.0.8
referencing 0.35.1
regex 2025.7.34
rembg 2.0.67
repath 0.9.0
requests 2.32.3
requests-file 2.1.0
requests-toolbelt 1.0.0
rf-clip 1.1
rich 13.0.1
rich-toolkit 0.14.9
rignore 0.6.4
rpds-py 0.20.0
rsa 4.9.1
ruamel.yaml 0.18.6
ruamel.yaml.clib 0.2.8
ruff 0.12.7
s3transfer 0.10.4
safetensors 0.6.1
sageattention 2.2.0 /app/SageAttention
scikit_build_core 0.11.5
scikit-image 0.24.0
scikit-learn 1.7.1
scipy 1.16.1
seaborn 0.13.2
segment-anything 1.0
sentencepiece 0.2.0
sentry-sdk 2.34.1
setproctitle 1.3.6
setuptools 79.0.1
setuptools-scm 8.3.1
sgl-kernel 0.3.2
shapely 2.0.7
shellingham 1.5.4
six 1.16.0
slack_sdk 3.33.5
smmap 5.0.2
sniffio 1.3.1
sortedcontainers 2.4.0
sounddevice 0.5.2
soundfile 0.13.1
soupsieve 2.5
soxr 0.5.0.post1
spandrel 0.4.1
SQLAlchemy 2.0.43
stack-data 0.6.2
starlette 0.37.2
structlog 24.4.0
supervision 0.26.1
sympy 1.13.1
threadpoolctl 3.6.0
tifffile 2025.6.11
tiktoken 0.10.0
timm 0.6.13
tldextract 5.1.3
tokenizers 0.21.4
toml 0.10.2
tomli 2.0.1
torch 2.5.1+cu124
torchaudio 2.5.1+cu124
torchelastic 0.2.2
torchscale 0.3.0
torchsde 0.2.6
torchvision 0.20.1+cu124
tqdm 4.66.5
traitlets 5.14.3
trampoline 0.1.2
transformers 4.55.0
transparent-background 1.3.3
triton 3.1.0
truststore 0.9.2
twilio 9.3.8
typeguard 4.3.0
typer 0.12.5
typer-config 1.4.2
types-dataclasses 0.6.6
typing_extensions 4.12.2
typing-inspect 0.9.0
typing-inspection 0.4.1
tzdata 2025.2
tzlocal 5.3.1
ultralytics 8.3.40
ultralytics-thop 2.0.15
unpaddedbase64 2.1.0
uritemplate 4.2.0
urllib3 2.2.3
uv 0.8.5
uvicorn 0.35.0
uvloop 0.21.0
virtualenv 20.33.1
wait_for2 0.3.2
wandb 0.21.1
watchfiles 1.1.0
wcwidth 0.2.13
websockets 15.0.1
wget 3.2
wheel 0.44.0
xgrammar 0.1.21
yapf 0.43.0
yarl 1.20.1
zipp 3.20.2
zstandard 0.23.0
zxing-cpp 2.2.0
