Video-LLaMA
Video-LLaMA copied to clipboard
Published
20 hours ago •
DAMO-NLP-SG
DAMO-NLP-SG
inferece如何使用多张V100代替一张A100？
你好，如果只有V100机器，加载llama13B版本，会OOM，但有多张V100，如何实现类似automap的功能，将模型映射到多张v100GPU上？
if ckpt_path:
print("Load first Checkpoint: {}".format(ckpt_path))
ckpt = torch.load(ckpt_path, map_location="cpu")
msg = model.load_state_dict(ckpt['model'], strict=False)
ckpt_path_2 = cfg.get("ckpt_2", "")
if ckpt_path_2:
print("Load second Checkpoint: {}".format(ckpt_path_2))
ckpt = torch.load(ckpt_path_2, map_location="cpu")
msg = model.load_state_dict(ckpt['model'], strict=False)
LlamaForCausalLM.from_pretrained的参数device_map改成auto后，上边load_state_dict过程，依旧load到一张卡导致OOM吧？
