DeepSpeedExamples
DeepSpeedExamples copied to clipboard
microsoft
Example models using DeepSpeed
https://github.com/microsoft/DeepSpeedExamples/blob/bae2667824974ac13dac28712462c14a2e594150/applications/DeepSpeed-Chat/training/utils/model/reward_model.py#L103 What if we change the loss to torch.log(torch.sigmoid(c_truncated_reward.mean() - r_truncated_reward.mean())) Instead of torch.log(torch.sigmoid(c_truncated_reward - r_truncated_reward)).mean()? I think in InstructGPT paper, it should be the latter?
add load downloaded huggingface dataset, in case of machines cannot connect to huggingface.co
Yuren(羽人) is a multi-language instructional dataset primarily in Chinese and English, suitable for DeepspeedChat. https://huggingface.co/datasets/pleisto/yuren
#!/bin/bash # Copyright (c) Microsoft Corporation. # SPDX-License-Identifier: Apache-2.0 # DeepSpeed Team # Note that usually LoRA needs to use larger learning rate OUTPUT_PATH=/mnt/bn/simple-nas/mlx/users/zhangyawei.ywsq/playground/arnold_ywsq/DeepSpeedExamples/applications/DeepSpeed-Chat/save/actor-models/7b1_bloom_lora mkdir -p $OUTPUT_PATH deepspeed --master_port 25104...
Hi, I tried using your DeepSpeed-chat example to train one `facebook/opt-1.3B` model using RLHF. I'm using a custom dataset of 500 examples. I updated the `data_utils.py` and `raw_datasets.py` files to...
I tried to reproduce the 13B rlhf training in A100-80GB * 8. I found the default training script here https://github.com/microsoft/DeepSpeedExamples/blob/master/applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/training_scripts/single_node/run_13b.sh where the per_device_train_batch_size and per_device_mini_train_batch_size is 16, which is different...
