Brian Qu

https://github.com/Coobiw/MiniGPT4Qwen/blob/master/lavis/models/minigpt4qwen_models/minigpt4qwen.py#L150 把这里hard code成`device_map="cuda"`试一下吧

可能暂时不考虑跟进，后续有空可以open出一个版本

moviepy的VideoFileClip有时候读取会有问题，所以我直接跳过了，大概只会浪费1k左右的视频（总共100k），所以就还好 如果你想尽可能避免的话，可以把[https://github.com/Coobiw/MPP-LLaVA/blob/master/lavis/datasets/datasets/video_instructions.py#L28](https://github.com/Coobiw/MPP-LLaVA/blob/master/lavis/datasets/datasets/video_instructions.py#L28)这个函数，参考[https://github.com/Coobiw/MPP-LLaVA/blob/master/webui_demo.py#L23](https://github.com/Coobiw/MPP-LLaVA/blob/master/webui_demo.py#L23)进行修改

是解决了吗，没有的话可以发一下配置的yaml文件，然后也可以检查下权重是否下载完全

看了下，应该不是报错，https://github.com/Coobiw/MiniGPT4Qwen/blob/master/lavis/models/base_model.py#L53 ``` msg = self.load_state_dict(state_dict, strict=False) logging.info("Missing keys {}".format(msg.missing_keys)) ``` 这里只是一个log，因为model.pth里只有中间的projection层的参数（第一步pretrain stage只训练中间projection层），其他都用的eva,blip2_qformer和qwen-7B的参数，所以会有一个INFO的提醒，按照 https://github.com/Coobiw/MiniGPT4Qwen/blob/master/WEIGHT.md 中下载权重就好 担心的话先训几步看看loss，显存不够的话可以把`freeze_llm`设为`True`先看下

@xianshunw No, mismatching wil not appear. The `batch` is a tuple, `batch[0]` is the input of the model and `batch[1]` is the label for computing loss. This is the protocol...

@xianshunw @avicooper1 Setting `shuffle=True` will not cause anything error in my experiment. I am just curious why it is `False` for default in Pipeline Dataloader Sampler.

@xianshunw Yeah, I've used the following sampler for my custom dataloader: ```python sampler = torch.utils.data.distributed.DistributedSampler( datasets['train'], num_replicas=engine.dp_world_size, rank=engine.mpu.get_data_parallel_rank(), shuffle=True ) ``` There is no problem.

same error when using single node multi gpus. Have you solve this problem? Looking forward to your reply!

Hi, I've solved this problem. For my case, this is a NCCL-Error. Just re-compile the NCCL please. Please refer to [this CN blog](https://blog.csdn.net/kabuto_hui/article/details/145949489). Then it's OK for me. BTW, I'll...