modelscope
DDP in DPO tuning on MLLM
我在使用DPO训练MLLM时，warning：Parameter 'function'=<bound method DPOTrainer.tokenize_row of <swift.trainers.dpo_trainer.DPOTrainer object at 0x7fa03ea4a850>> of the transform datasets.arrow_dataset.Dataset._map_single couldn't be hashed properly, a random hash was used instead. Make sure your transforms and parameters are serializable with pickle or dill for the dataset fingerprinting and caching to work. If you reuse this transform, the caching mechanism will consider it to be different from the previous calls and recompute everything. This warning is only showed once. Subsequent hashing failures won't be showed. 这会造成DDP时数据加载效率降低，当数据较多时NCCL的同步出现问题（超时）
而且我在训练的时候发现在加载数据集的时候，他会把图片或者视频一起加载上去，这导致内存消耗变得巨大，不知道作者能不能改一下逻辑，改成原本sft的那种逻辑呢
第一个问题在于bfloat16的模型权重无法被序列化, float16精度下应该没有这个问题
第二个问题可以尝试下新支持的流式数据加载, --streaming true
