InternVL-C(ontrastive) 和 InternVL-G(enerative) 如何做多GPU推理

[lisenjie757]

直接用device_map="auto" 或 device_map="balanced_low_0" 会报错，text_embeds = text_embeds[torch.arange(text_embeds.shape[0]), attention_mask.sum(1) - 1] RuntimeError: indices should be either on cpu or on the same device as the indexed tensor (cuda:1)