OpenTalker
webUi推理失败，未找到audio2pose_checkpoint
had the same issue any solutions? running on amd
WARNING: The new version of the model will be updated by safetensor, you may need to download it mannully. We run the old version of the checkpoint this time! {'wav2lip_checkpoint': '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\wav2lip.pth', 'audio2pose_checkpoint': '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\auido2pose_00140-model.pth', 'audio2exp_checkpoint': '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\auido2exp_00300-model.pth', 'free_view_checkpoint': '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\facevid2vid_00189-model.pth.tar', 'path_of_net_recon_model': '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\epoch_20.pth', 'dir_of_BFM_fitting': 'C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker/src/config', 'audio2pose_yaml_path': 'C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker/src/config\auido2pose.yaml', 'audio2exp_yaml_path': 'C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker/src/config\auido2exp.yaml', 'use_safetensor': False, 'mappingnet_checkpoint': '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\mapping_00109-model.pth.tar', 'facerender_yaml': 'C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker/src/config\facerender_still.yaml'} Traceback (most recent call last): File "C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker\src\test_audio2coeff.py", line 48, in init load_cpk(sadtalker_path['audio2pose_checkpoint'], model=self.audio2pose_model, device=device) File "C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker\src\test_audio2coeff.py", line 17, in load_cpk checkpoint = torch.load(checkpoint_path, map_location=torch.device(device)) File "C:\Users\Ghost\stable-diffusion-webui\modules\safe.py", line 108, in load return load_with_extra(filename, *args, extra_handler=global_extra_handler, **kwargs) File "C:\Users\Ghost\stable-diffusion-webui\modules\safe.py", line 156, in load_with_extra return unsafe_torch_load(filename, *args, **kwargs) File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 791, in load with _open_file_like(f, 'rb') as opened_file: File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 271, in _open_file_like return _open_file(name_or_buffer, mode) File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\torch\serialization.py", line 252, in init super().init(open(name, mode)) OSError: [Errno 22] Invalid argument: '"C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints"\auido2pose_00140-model.pth'
During handling of the above exception, another exception occurred:
Traceback (most recent call last): File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\gradio\routes.py", line 422, in run_predict output = await app.get_blocks().process_api( File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1323, in process_api result = await self.call_function( File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\gradio\blocks.py", line 1051, in call_function prediction = await anyio.to_thread.run_sync( File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\anyio\to_thread.py", line 31, in run_sync return await get_asynclib().run_sync_in_worker_thread( File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 937, in run_sync_in_worker_thread return await future File "C:\Users\Ghost\stable-diffusion-webui\venv\lib\site-packages\anyio_backends_asyncio.py", line 867, in run result = context.run(func, *args) File "C:\Users\Ghost\stable-diffusion-webui\modules\call_queue.py", line 13, in f res = func(*args, **kwargs) File "C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker\src\gradio_demo.py", line 49, in test self.audio_to_coeff = Audio2Coeff(self.sadtalker_paths, self.device) File "C:\Users\Ghost\stable-diffusion-webui/extensions/SadTalker\src\test_audio2coeff.py", line 50, in init raise Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint
also it appears that some of the "" are "/" backwards how would i go about fixing this?
set my batch file up like this on windows by rightclicking and editing for those that dont understand... so far the script is running/loading and operating as it should will post success if i develop a successful video. looks like an update has been made!
@echo off
set PYTHON="C:\Users\Ghost\AppData\Local\Programs\Python\Python310\python.exe"
set GIT=
set VENV_DIR=
set COMMANDLINE_ARGS=--precision full --no-half --skip-torch-cuda-test --disable-safe-unpickle
set SADTALKER_CHECKPOINTS=C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints
git pull
call webui.bat
edit - all works as it should on my end using an amd cpu, best of luck!
I ran to the same issue on windows. All model that I downloaded in `./checkpoints/' have some weird character at the end which is invisible in file explorer.
Click rename the file and delete that character if there is a space at the end. This solve my issue
I found out the error source is the src/utils/init_path.py In this code,seem like we need to find different models to download ,
def init_path(checkpoint_dir, config_dir, size=512, old_version=False, preprocess='crop'):
if old_version:
#### load all the checkpoint of `pth`
# gradio_demo里面调用的是这个
sadtalker_paths = {
'wav2lip_checkpoint' : os.path.join(checkpoint_dir, 'wav2lip.pth'),
'audio2pose_checkpoint' : os.path.join(checkpoint_dir, 'auido2pose_00140-model.pth'),
'audio2exp_checkpoint' : os.path.join(checkpoint_dir, 'auido2exp_00300-model.pth'),
'free_view_checkpoint' : os.path.join(checkpoint_dir, 'facevid2vid_00189-model.pth.tar'),
'path_of_net_recon_model' : os.path.join(checkpoint_dir, 'epoch_20.pth')
}
Including wav2lip ,audio2pose,audio2exp_checkpoint, This repo doesn't have it ?
fail loading checkpoint, so you need check if you had these checkpoint files, they are located at the directory such as "xxx\stable-diffusion-webui-master\webui\extensions\SadTalker\checkpoints". if not, you can dowload them all at the github "https://github.com/OpenTalker/SadTalker/releases/"
set my batch file up like this on windows by rightclicking and editing for those that dont understand... so far the script is running/loading and operating as it should will post success if i develop a successful video. looks like an update has been made!
@echo off set PYTHON="C:\Users\Ghost\AppData\Local\Programs\Python\Python310\python.exe" set GIT= set VENV_DIR= set COMMANDLINE_ARGS=--precision full --no-half --skip-torch-cuda-test --disable-safe-unpickle set SADTALKER_CHECKPOINTS=C:\Users\Ghost\stable-diffusion-webui\extensions\SadTalker\checkpoints git pull call webui.bat
edit - all works as it should on my end using an amd cpu, best of luck!
@ytnurg I have to respectfully disagree with your commandline args, cuda test should not be skipped. By doing that, you're only using your CPU and not GPU which is very efficient and fast.
I got it working fixing this issue, the only minor issue is the small display doesnt work but who cares its working. Here are the 5 secrets I learned.
I had many errors along the way, this was my last error. "Exception("Failed in loading audio2pose_checkpoint") Exception: Failed in loading audio2pose_checkpoint" Hope this helps out.
I used this older version: I might try the newer one later: "https://github.com/OpenTalker/SadTalker/tree/cd4c0465ae0b54a6f85af57f5c65fec9fe23e7f8"
#1 "https://raw.githubusercontent.com/Winfredy/SadTalker/main/scripts/download_models.sh" put this output in both the destination locations and this location too ".\SadTalker\SADTALKER_CHECKPOINTS" "https://sadtalkerai.com/how-to-solve-sadtalker-audio2pose-checkpoint-error/" this talks about SADTALKER_CHECKPOINTS folder
#2 "https://huggingface.co/vinthony/SadTalker/tree/main" these files are missing add them to this location ".\SadTalker\SADTALKER_CHECKPOINTS"
#3 shout out to sakura1217 for this piece of gold image. follow this too.
#4 Redundant files are keys, I am sure there is a method to the madness but better you have extra files and have it working. Plus don't forget to add a ".\SadTalker\results" folder important for your output or it wont work.
#5 If you arent sure and you are still having issues here are my Package/version. I am on Win 11 running Cuda Package Version
absl-py 2.1.0 addict 2.4.0 aiofiles 23.2.1 aiosignal 1.3.1 altair 5.4.1 annotated-types 0.7.0 anyio 4.4.0 attrs 24.2.0 audioread 3.0.1 av 12.3.0 babel 2.16.0 basicsr 1.4.2 better-profanity 0.7.0 bibtexparser 2.0.0b7 blinker 1.8.2 cachetools 5.5.0 certifi 2024.8.30 cffi 1.17.1 charset-normalizer 3.3.2 click 8.1.7 clip 1.0 clldutils 3.22.2 cn2an 0.5.22 colorama 0.4.6 coloredlogs 15.0.1 colorlog 6.8.2 contourpy 1.1.1 cycler 0.12.1 decorator 5.1.1 Deprecated 1.2.14 detoxify 0.5.2 dlinfo 1.2.1 einops 0.8.0 eng_to_ipa 0.0.2 exceptiongroup 1.2.2 face-alignment 1.3.5 facexlib 0.3.0 fastapi 0.114.2 ffmpeg-python 0.2.0 ffmpy 0.4.0 filelock 3.16.0 filterpy 1.4.5 flatbuffers 24.3.25 fonttools 4.53.1 frozenlist 1.4.1 fsspec 2024.9.0 ftfy 6.2.3 future 1.0.0 gfpgan 1.3.8 gitdb 4.0.11 GitPython 3.1.43 google-auth 2.34.0 google-auth-oauthlib 1.0.0 gradio 3.41.2 gradio_client 0.5.0 grpcio 1.66.1 h11 0.14.0 httpcore 1.0.5 httpx 0.27.2 huggingface-hub 0.24.7 humanfriendly 10.0 idna 3.10 imageio 2.19.3 imageio-ffmpeg 0.4.7 importlib_metadata 8.5.0 importlib_resources 6.4.5 isodate 0.6.1 jaconv 0.4.0 Jinja2 3.1.4 joblib 1.1.0 jsonschema 4.23.0 jsonschema-specifications 2023.12.1 kiwisolver 1.4.7 kornia 0.6.8 kornia_rs 0.1.5 langid 1.1.6 language-tags 1.2.0 lazy_loader 0.4 librosa 0.9.2 linkify-it-py 2.0.3 llvmlite 0.41.1 lmdb 1.5.1 Markdown 3.7 markdown-it-py 3.0.0 MarkupSafe 2.1.5 matplotlib 3.7.5 mdurl 0.1.2 more-itertools 10.5.0 msgpack 1.0.8 multidict 6.0.5 music-tag 0.4.3 mutagen 1.47.0 narwhals 1.8.1 networkx 3.1 numba 0.58.1 numpy 1.23.4 oauthlib 3.2.2 openai-whisper 20231117 opencv-python 4.10.0.84 orjson 3.10.7 packaging 24.1 pandas 2.0.3 phonemizer 3.3.0 pillow 10.4.0 pip 23.0.1 pkgutil_resolve_name 1.3.10 platformdirs 4.3.3 pooch 1.8.2 proces 0.1.7 progressbar 2.5 protobuf 5.28.1 pyarrow 17.0.0 pyasn1 0.6.1 pyasn1_modules 0.4.1 pycparser 2.22 pydantic 2.9.1 pydantic_core 2.23.3 pydeck 0.9.1 pydub 0.25.1 Pygments 2.18.0 pylatexenc 2.10 pyparsing 3.1.4 pyreadline3 3.4.1 python-dateutil 2.9.0.post0 python-multipart 0.0.9 pytz 2024.2 PyWavelets 1.4.1 PyYAML 6.0.2 rdflib 7.0.0 referencing 0.35.1 regex 2024.9.11 requests 2.32.3 requests-oauthlib 2.0.0 resampy 0.3.1 rich 13.8.1 rpds-py 0.10.0 rsa 4.9 ruff 0.6.5 safetensors 0.4.5 scikit-image 0.19.3 scikit-learn 1.1.3 scipy 1.10.1 segments 2.2.1 semantic-version 2.10.0 sentencepiece 0.2.0 setuptools 56.0.0 shellingham 1.5.4 six 1.16.0 smmap 5.0.1 sniffio 1.3.1 soundfile 0.12.1 spandrel 0.3.4 starlette 0.38.5 streamlit 1.38.0 tabulate 0.9.0 taming-transformers 0.0.1 tb-nightly 2.14.0a20230808 tenacity 8.5.0 tensorboard-data-server 0.7.2 threadpoolctl 3.5.0 tifffile 2023.7.10 tiktoken 0.7.0 tokenizers 0.19.1 toml 0.10.2 tomli 2.0.1 tomlkit 0.12.0 torch 1.12.1+cu113 torchaudio 0.12.1+cu113 torchlibrosa 0.1.0 torchsde 0.2.6 torchvision 0.13.1+cu113 tornado 6.4.1 tqdm 4.66.5 trampoline 0.1.2 transformers 4.44.2 typer 0.12.5 typing_extensions 4.12.2 tzdata 2024.1 uc-micro-py 1.0.3 uritemplate 4.1.1 urllib3 2.2.3 uvicorn 0.30.6 voicefixer 0.1.3 watchdog 4.0.2 wavmark 0.0.3 wcwidth 0.2.13 websockets 11.0.3 Werkzeug 3.0.4 wheel 0.44.0 yacs 0.1.8 yapf 0.40.2 zipp 3.20.2
[notice] A new release of pip is available: 23.0.1 -> 24.2 [notice] To update, run: python.exe -m pip install --upgrade pip
