text_generation_webui部署的几个问题

[yisampi]

1.FileNotFoundError: [Errno 2] No such file or directory: './models/visualcla_merged-7b/pytorch_model.bin' 对于合并权重的情况 需要cp visualcla/pytorch_model.bin models/visualcla_merged-7b/ 不知道这样对不对

2.OSError: Can't load the configuration of './models/visualcla_merged-7b/vision_encoder'. If you were trying to load it from 'https://huggingface.co/models', make sure you don't have a local directory with the same name. Otherwise, make sure './models/visualcla_merged-7b/vision_encoder' is the correct path to a directory containing a config.json file 对于合并权重的情况 需要cp -r ./visualcla/vision_encoder/ ./models/visualcla_merged-7b/ 不知道这样对不对

3.OSError: ./models/visualcla_merged-7b does not appear to have a file named preprocessor_config.json. Checkout 'https://huggingface.co/./models/visualcla_merged-7b/main' for available files. 对于合并权重的情况 cp ./visualcla/preprocessor_config.json models/visualcla_merged-7b/ 不知道这样对不对

4.KeyError: 'visual_resampler_config' 以上操作完了之后，重新运行server.py $ python server.py --model=visualcla_merged-7b --multimodal-pipeline=visualcla-7b --chat --settings=settings-visualcla.yaml --share --load-in-8bit 2023-07-27 09:31:45 WARNING:The gradio "share link" feature uses a proprietary executable to create a reverse tunnel. Use it with care. 2023-07-27 09:31:47 INFO:Loading settings from settings-visualcla.yaml... 2023-07-27 09:31:47 INFO:Loading visualcla_merged-7b... 2023-07-27 09:38:36 WARNING:models/visualcla_merged-7b/special_tokens_map.json is different from the original LlamaTokenizer file. It is either customized or outdated. 2023-07-27 09:38:36 INFO:Loaded the model in 408.25 seconds.

2023-07-27 09:38:36 INFO:Loading the extension "multimodal"... 2023-07-27 09:38:36 INFO:VisualCLA - Loading CLIP from ./models/visualcla_merged-7b/vision_encoder as torch.float32 on cuda:0... 2023-07-27 09:38:38 INFO:VisualCLA - Loading visual resampler from ./models/visualcla_merged-7b/ as torch.float32 on cuda:0... Traceback (most recent call last): File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/server.py", line 1179, in create_interface() File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/server.py", line 1086, in create_interface extensions_module.create_extensions_block() File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/modules/extensions.py", line 175, in create_extensions_block extension.ui() File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/script.py", line 119, in ui multimodal_embedder = MultimodalEmbedder(params) File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/multimodal_embedder.py", line 27, in init pipeline, source = load_pipeline(params) File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/pipeline_loader.py", line 30, in load_pipeline pipeline = getattr(pipeline_modules[k], 'get_pipeline')(shared.args.multimodal_pipeline, params) File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/pipelines/visualcla/pipelines.py", line 11, in get_pipeline return VisualCLA_7B_Pipeline(params) File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/pipelines/visualcla/visualcla.py", line 140, in init super().init(params) File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/pipelines/visualcla/visualcla.py", line 30, in init self.image_processor, self.vision_tower, self.visual_resampler, self.image_projection_layer = self._load_models() File "/home/yibo/text-generation-webui-Visual-Chinese-LLaMA-Alpaca/extensions/multimodal/pipelines/visualcla/visualcla.py", line 47, in _load_models visual_resampler_config = VisualResamplerConfig.from_dict(json.load(open(os.path.join(shared.settings['visualcla_merged_model'], 'config.json')))['visual_resampler_config']) KeyError: 'visual_resampler_config'

配置文件config.json如下 more models/visualcla_merged-7b/config.json { "_name_or_path": "chinese-alpaca-plus-7b/", "architectures": [ "LlamaForCausalLM" ], "bos_token_id": 1, "eos_token_id": 2, "hidden_act": "silu", "hidden_size": 4096, "initializer_range": 0.02, "intermediate_size": 11008, "max_position_embeddings": 2048, "model_type": "llama", "num_attention_heads": 32, "num_hidden_layers": 32, "pad_token_id": 0, "rms_norm_eps": 1e-06, "tie_word_embeddings": false, "torch_dtype": "float16", "transformers_version": "4.30.2", "use_cache": true, "vocab_size": 49954 }

请帮忙看看，谢谢

[GoGoJoestar]

关于1、2、3：对于合并后的权重，需要将文本端权重text_encoder目录（即你的visualcla/text_encoder目录）复制到text-generation-webui目录的models下并命名为visualcla_merged-7b，然后将tokenizer相关的四个文件复制到models/visualcla_merged-7b下，最后修改text-generation-webui的settings-visualcla.yaml中visualcla_merged_model的值为合并后权重的目录（即你的visualcla目录）。 models/visualcla_merged-7b目录下只包含文本端相关文件，不需要把vision_encodel和resampler等部分放进去。其余部分只需要在settings-visualcla.yaml中设置好目录就可以读取，看上面的报错可能是这里设置的目录不对

[yisampi]

嗯 是我配置错了 visualcla_merged_model : [Path/For/Merged_Model] 之前这里路径我配置成models/visualcla_merged-7b，应该是Merged_Model的路径 谢谢