zhongzee

[BUG]deepspeed-chat training error on v100 * 8:RuntimeError: output tensor must have the same type as input tensor

1

Traceback (most recent call last): File "/mnt/afs/liwenhao/wuzhongze/mPLUG-Owl-main/mPLUG-Owl2/mplug_owl2/train/train_xformers.py", line 13, in train() File "/mnt/afs/liwenhao/wuzhongze/mPLUG-Owl-main/mPLUG-Owl2/mplug_owl2/train/train.py", line 778, in train trainer.train() File "/mnt/afs/liwenhao/mplug_owl/lib/python3.10/site-packages/transformers/trainer.py", line 1539, in train return inner_training_loop( File "/mnt/afs/liwenhao/mplug_owl/lib/python3.10/site-packages/transformers/trainer.py", line 1809,...

[Usage] deepspeed-chat training error on v100 * 8:RuntimeError: output tensor must have the same type as input tensor

4

### Describe the issue My zero3.json config is: { "fp16": { "enabled": “auto”, "loss_scale": 0, "loss_scale_window": 1000, "initial_scale_power": 16, "hysteresis": 2, "min_loss_scale": 1 }, "bf16": { "enabled":“auto” }, "train_micro_batch_size_per_gpu": "auto",...

Table7（OV_lvis）的结果应该如何复现，配置文件如何设置呢？

31

目前configs下只有finetune-coco和zero-shot的配置文件，请问如下结果应该怎么配置configs文件呢？训练和测试类别数应该怎么设置 

如何分布式训练？

1

chmod +x tools/dist_train.sh # sample command for pre-training, use AMP for mixed-precision training ./tools/dist_train.sh configs/pretrain/yolo_world_l_t2i_bn_2e-4_100e_4x8gpus_obj365v1_goldg_train_lvis_minival.py 8 --amp 这个命令只适合单服务器的，如和改成多服务器的形式呢？nodes和node_rank要怎么设置呢？

如何冻结预训练权重，进行部分微调？

2

yolo_world_v2_l_clip_large_vlpan_bn_2e-3_100e_4x8gpus_obj365v1_goldg_train_800ft_lvis_minival.py 部分配置如下：如果想要加载该预训练权重，并且执行dist_train.sh 1）我应该怎么冻结下面load_from的的所有权重，并修改backbone部分结构，比如添加适配层来微调该权重呢？ 2）load_from的权重包括yolov8backbone/neck层/head层的权重是吗？ load_from = '/mnt/afs/huangtao3/wzz/YOLO-World/weights/YOLO-World/yolo_world_v2_l_clip_large_o365v1_goldg_pretrain_800ft-9df82e55.pth' text_model_name = '/mnt/afs/huangtao3/wzz/YOLO-World/weights/clip-vit-large-patch14-336' img_scale = (800, 800) # model settings model = dict( type='YOLOWorldDetector', mm_neck=True, num_train_classes=num_training_classes, num_test_classes=num_classes, data_preprocessor=dict(type='YOLOWDetDataPreprocessor'), backbone=dict( _delete_=True, type='MultiModalYOLOBackbone', image_model={{_base_.model.backbone}}, text_model=dict(...