AIGC-Audio
No such file or directory: 'useful_ckpts/CLAP/CLAP_weights_2022.pth'
When running the Python script in Windows (and WSL2), I keep getting this error:
FileNotFoundError: [Errno 2] No such file or directory: 'useful_ckpts/CLAP/CLAP_weights_2022.pth'
Those files don't exist in my installation. How do I get them?
Looks like this Hugging Face Space does not exist:
https://huggingface.co/spaces/DiffusionSpeech/
And that's where it is trying to download from. No trace of it.
@RayeRen @alefiury @Guseyn013 @geekinchief22 @simpleoier
guys ，have you find the way out?
you can download these files in https://huggingface.co/spaces/AIGC-Audio/AudioGPT
wget -P text_to_audio/Make_An_Audio/useful_ckpts/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/ta40multi_epoch=000085.ckpt
wget -P text_to_audio/Make_An_Audio/useful_ckpts/CLAP/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/CLAP/CLAP_weights_2022.pth
wget -P text_to_audio/Make_An_Audio/useful_ckpts/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/ta54_epoch=000216.ckpt
wget -P text_to_audio/Make_An_Audio/useful_ckpts/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/inpaint7_epoch00047.ckpt
wget -P text_to_audio/Make_An_Audio/vocoder/logs/bigv16k53w/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/bigv16k53w/args.yml https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/bigv16k53w/best_netG.pt
wget -P text_to_audio/Make_An_Audio/vocoder/logs/hifi_0127/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/hifi_0127/args.yml https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/hifi_0127/best_netG.pt
Would be cool to get it to run with ProDiff maybe: https://github.com/Rongjiehuang/ProDiff
i had to hardcode the path in txt2audio_args.yaml file to get this to work.
i had to hardcode the path in txt2audio_args.yaml file to get this to work.
In my code the path is already there, what else can I do?
wget -P text_to_audio/Make_An_Audio/useful_ckpts/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/ta40multi_epoch=000085.ckpt wget -P text_to_audio/Make_An_Audio/useful_ckpts/CLAP/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/CLAP/CLAP_weights_2022.pth wget -P text_to_audio/Make_An_Audio/useful_ckpts/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/ta54_epoch=000216.ckpt wget -P text_to_audio/Make_An_Audio/useful_ckpts/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/useful_ckpts/inpaint7_epoch00047.ckpt wget -P text_to_audio/Make_An_Audio/vocoder/logs/bigv16k53w/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/bigv16k53w/args.yml https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/bigv16k53w/best_netG.pt wget -P text_to_audio/Make_An_Audio/vocoder/logs/hifi_0127/ https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/hifi_0127/args.yml https://huggingface.co/spaces/AIGC-Audio/AudioGPT/resolve/main/text_to_audio/Make_An_Audio/vocoder/logs/hifi_0127/best_netG.pt
This didn't work for me
Longer error log - what's the fix?
Working with z of shape (1, 4, 106, 106) = 44944 dimensions.
making attention of type 'vanilla' with 512 in_channels
making attention of type 'vanilla' with 512 in_channels
making attention of type 'vanilla' with 512 in_channels
making attention of type 'vanilla' with 512 in_channels
making attention of type 'vanilla' with 256 in_channels
making attention of type 'vanilla' with 256 in_channels
making attention of type 'vanilla' with 256 in_channels
╭─────────────────────────────── Traceback (most recent call last) ────────────────────────────────╮
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/audio- │
│ chatgpt.py:1377 in <module> │
│ │
│ 1374 │
│ 1375 │
│ 1376 if __name__ == '__main__': │
│ ❱ 1377 │ bot = ConversationBot() │
│ 1378 │ with gr.Blocks(css="#chatbot .overflow-y-auto{height:500px}") as demo: │
│ 1379 │ │ with gr.Row(): │
│ 1380 │ │ │ gr.Markdown("## AudioGPT") │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/audio- │
│ chatgpt.py:1057 in __init__ │
│ │
│ 1054 │ │ self.llm = OpenAI(temperature=0) │
│ 1055 │ │ self.t2i = T2I(device="cuda:0") │
│ 1056 │ │ self.i2t = ImageCaptioning(device="cuda:0") │
│ ❱ 1057 │ │ self.t2a = T2A(device="cuda:0") │
│ 1058 │ │ self.tts = TTS(device="cpu") │
│ 1059 │ │ self.t2s = T2S(device="cpu") │
│ 1060 │ │ self.i2a = I2A(device="cuda:0") │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/audio- │
│ chatgpt.py:144 in __init__ │
│ │
│ 141 │ def __init__(self, device): │
│ 142 │ │ print("Initializing Make-An-Audio to %s" % device) │
│ 143 │ │ self.device = device │
│ ❱ 144 │ │ self.sampler = self._initialize_model('text_to_audio/Make_An_Audio/configs/text_ │
│ 145 │ │ self.vocoder = VocoderBigVGAN('text_to_audio/Make_An_Audio/vocoder/logs/bigv16k5 │
│ 146 │ │
│ 147 │ def _initialize_model(self, config, ckpt, device): │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/audio- │
│ chatgpt.py:149 in _initialize_model │
│ │
│ 146 │ │
│ 147 │ def _initialize_model(self, config, ckpt, device): │
│ 148 │ │ config = OmegaConf.load(config) │
│ ❱ 149 │ │ model = instantiate_from_config(config.model) │
│ 150 │ │ model.load_state_dict(torch.load(ckpt, map_location='cpu')["state_dict"], strict │
│ 151 │ │ │
│ 152 │ │ model = model.to(device) │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/text_t │
│ o_audio/Make_An_Audio/ldm/util.py:118 in instantiate_from_config │
│ │
│ 115 │ │ elif config == "__is_unconditional__": │
│ 116 │ │ │ return None │
│ 117 │ │ raise KeyError("Expected key `target` to instantiate.") │
│ ❱ 118 │ return get_obj_from_str(config["target"],reload=reload)(**config.get("params", dict( │
│ 119 │
│ 120 │
│ 121 def get_obj_from_str(string, reload=False): │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/text_t │
│ o_audio/Make_An_Audio/ldm/models/diffusion/ddpm_audio.py:76 in __init__ │
│ │
│ 73 │ │ else: │
│ 74 │ │ │ self.register_buffer('scale_factor', torch.tensor(scale_factor)) │
│ 75 │ │ self.instantiate_first_stage(first_stage_config) │
│ ❱ 76 │ │ self.instantiate_cond_stage(cond_stage_config) │
│ 77 │ │ self.cond_stage_forward = cond_stage_forward │
│ 78 │ │ self.clip_denoised = False │
│ 79 │ │ self.bbox_tokenizer = None │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/text_t │
│ o_audio/Make_An_Audio/ldm/models/diffusion/ddpm_audio.py:134 in instantiate_cond_stage │
│ │
│ 131 │ │ │ │ self.cond_stage_model = None │
│ 132 │ │ │ │ # self.be_unconditional = True │
│ 133 │ │ │ else: │
│ ❱ 134 │ │ │ │ model = instantiate_from_config(config) │
│ 135 │ │ │ │ self.cond_stage_model = model.eval() │
│ 136 │ │ │ │ self.cond_stage_model.train = disabled_train │
│ 137 │ │ │ │ for param in self.cond_stage_model.parameters(): │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/text_t │
│ o_audio/Make_An_Audio/ldm/util.py:118 in instantiate_from_config │
│ │
│ 115 │ │ elif config == "__is_unconditional__": │
│ 116 │ │ │ return None │
│ 117 │ │ raise KeyError("Expected key `target` to instantiate.") │
│ ❱ 118 │ return get_obj_from_str(config["target"],reload=reload)(**config.get("params", dict( │
│ 119 │
│ 120 │
│ 121 def get_obj_from_str(string, reload=False): │
│ │
│ /mnt/batch/tasks/shared/LS_root/mounts/clusters/azure1/code/Users/azure/audiogpt/AudioGPT/text_t │
│ o_audio/Make_An_Audio/ldm/modules/encoders/modules.py:178 in __init__ │
│ │
│ 175 │ def __init__(self, weights_path, freeze=True, device="cuda", max_length=77): # clip │
│ 176 │ │ super().__init__() │
│ 177 │ │ │
│ ❱ 178 │ │ model_state_dict = torch.load(weights_path, map_location=torch.device('cpu'))['m │
│ 179 │ │ match_params = dict() │
│ 180 │ │ for key in list(model_state_dict.keys()): │
│ 181 │ │ │ if 'caption_encoder' in key: │
│ │
│ /anaconda/envs/azureml_py38/lib/python3.8/site-packages/torch/serialization.py:699 in load │
│ │
│ 696 │ if 'encoding' not in pickle_load_args.keys(): │
│ 697 │ │ pickle_load_args['encoding'] = 'utf-8' │
│ 698 │ │
│ ❱ 699 │ with _open_file_like(f, 'rb') as opened_file: │
│ 700 │ │ if _is_zipfile(opened_file): │
│ 701 │ │ │ # The zipfile reader is going to advance the current file position. │
│ 702 │ │ │ # If we want to actually tail call to torch.jit.load, we need to │
│ │
│ /anaconda/envs/azureml_py38/lib/python3.8/site-packages/torch/serialization.py:230 in │
│ _open_file_like │
│ │
│ 227 │
│ 228 def _open_file_like(name_or_buffer, mode): │
│ 229 │ if _is_path(name_or_buffer): │
│ ❱ 230 │ │ return _open_file(name_or_buffer, mode) │
│ 231 │ else: │
│ 232 │ │ if 'w' in mode: │
│ 233 │ │ │ return _open_buffer_writer(name_or_buffer) │
│ │
│ /anaconda/envs/azureml_py38/lib/python3.8/site-packages/torch/serialization.py:211 in __init__ │
│ │
│ 208 │
│ 209 class _open_file(_opener): │
│ 210 │ def __init__(self, name, mode): │
│ ❱ 211 │ │ super(_open_file, self).__init__(open(name, mode)) │
│ 212 │ │
│ 213 │ def __exit__(self, *args): │
│ 214 │ │ self.file_like.close() │
╰──────────────────────────────────────────────────────────────────────────────────────────────────╯
FileNotFoundError: [Errno 2] No such file or directory: 'useful_ckpts/CLAP/CLAP_weights_2022.pth'
The simple and easy way to solve this issue is to download these files from here
create a folder useful_ckpts in main directory as same as within the link. I hope this will resolve the issues. There are certain issues which will be faced after resolving this issue, so the better option would be to clone the full from Hugging Face with source code.
python audio-chatgpt.py Initializing AudioGPT Initializing Make-An-Audio to cpu LatentDiffusion_audio: Running in eps-prediction mode DiffusionWrapper has 160.22 M params. making attention of type 'vanilla' with 256 in_channels making attention of type 'vanilla' with 256 in_channels making attention of type 'vanilla' with 512 in_channels making attention of type 'vanilla' with 512 in_channels making attention of type 'vanilla' with 512 in_channels Working with z of shape (1, 4, 106, 106) = 44944 dimensions. making attention of type 'vanilla' with 512 in_channels making attention of type 'vanilla' with 512 in_channels making attention of type 'vanilla' with 512 in_channels making attention of type 'vanilla' with 512 in_channels making attention of type 'vanilla' with 256 in_channels making attention of type 'vanilla' with 256 in_channels making attention of type 'vanilla' with 256 in_channels Some weights of the model checkpoint at bert-base-uncased were not used when initializing BertModel: ['cls.predictions.bias', 'cls.predictions.transform.dense.weight', 'cls.predictions.transform.LayerNorm.bias', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.weight']
- This IS expected if you are initializing BertModel from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).
- This IS NOT expected if you are initializing BertModel from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).
TextEncoder comes with 111.32 M params.
Traceback (most recent call last):
File "audio-chatgpt.py", line 1378, in
bot = ConversationBot() File "audio-chatgpt.py", line 1057, in init self.t2a = T2A(device="cpu") File "audio-chatgpt.py", line 144, in init self.sampler = self._initialize_model('text_to_audio/Make_An_Audio/configs/text_to_audio/txt2audio_args.yaml', 'text_to_audio/Make_An_Audio/useful_ckpts/ta40multi_epoch=000085.ckpt', device=device) File "audio-chatgpt.py", line 150, in _initialize_model model.load_state_dict(torch.load(ckpt, map_location='cpu')["state_dict"], strict=False) File "/root/anaconda3/envs/audiogpt/lib/python3.8/site-packages/torch/serialization.py", line 713, in load return _legacy_load(opened_file, map_location, pickle_module, **pickle_load_args) File "/root/anaconda3/envs/audiogpt/lib/python3.8/site-packages/torch/serialization.py", line 920, in _legacy_load magic_number = pickle_module.load(f, **pickle_load_args) _pickle.UnpicklingError: invalid load key, '<'.
can run in cpu????
