模型无法转换怎么办？

[alfonsocheng1025]

模型无法转换怎么办？

02/03 10:46:39 - mmengine - INFO - Load PTH model from /private/xtuner/work_dirs/llava_llama3_8b_instruct_clip_vit_large_p14_336_e1_gpu8_sharegpt4v_pretrain/iter_7038.pth 02/03 10:46:39 - mmengine - INFO - Convert LLM to float16 Traceback (most recent call last): File "/private/xtuner/xtuner/tools/model_converters/pth_to_hf.py", line 142, in main() File "/private/xtuner/xtuner/tools/model_converters/pth_to_hf.py", line 130, in main model.to_hf( File "/private/xtuner/xtuner/model/llava.py", line 343, in to_hf self.to_xtuner_llava(cfg, save_dir, fp32, save_pretrained_kwargs) File "/private/xtuner/xtuner/model/llava.py", line 361, in to_xtuner_llava self.llm.half() File "/opt/conda/envs/PyTorch-2.2.2/lib/python3.10/site-packages/transformers/modeling_utils.py", line 3115, in half raise ValueError( ValueError: .half() is not supported for quantized model. Please use the model as it is, since the model has already been casted to the correct dtype.

[happy-for-you-every]

1. 修改模型权重加载

def parse_model_states(files, dtype=DEFAULT_DTYPE): zero_model_states = [] for file in files: # Modify model weight loading state_dict = torch.load(file, map_location='cpu', weights_only=False) # 添加参数 # ...后续处理...

2. 涉及优化器状态加载，同样需修改

@torch.no_grad() def parse_optim_states(files, ds_checkpoint_dir, dtype=DEFAULT_DTYPE): zero_stage = None world_size = None total_files = len(files) flat_groups = [] torch.serialization.add_safe_globals([ConfigDict]) for f in tqdm(files, desc="Load Checkpoints"): state_dict = torch.load(f, map_location=device, weights_only=False) # 添加参数 # ...后续处理...