DeepSpeedExamples

DeepSpeedExamples copied to clipboard

reward model implementation

4

In the reward model implementation, I noticed these two lines of code， ` c_truncated_reward = chosen_reward[divergence_ind:end_ind] r_truncated_reward = rejected_reward[divergence_ind:end_ind]` It should take the answer part, but chosen and rejected take...

DamonYangyang

DeepSpeed-Chat: prefetch of layers during reward model forward pass leads to error during sample generation

18

When running step 3 with ZERO stage 3 enabled for both the actor and critic models, I get the following error (line numbers may be offset due to debug statements...

adammoody

【BUG】occur error：AttributerError：'DeepSpeedHybridEngine' object has no attribute 'mp_group' whiling run llama7b for step3/rlhf/ppo

3


Pattaro

Finished training but the inference performance doesn't look good

4

Hi, I have finished training the following models: facebook/opt-1.3b (step 1,2 and 3) facebook/opt-6.7b (step 1) **Here is the performance shown at the bottom of the chatbot.py script:** ``` Human:...

alibabadoufu

Bug in model save with Zero stage 3

1

``` File "main.py", line 334, in main save_hf_format(model, tokenizer, args) File ".../applications/DeepSpeed-Chat/training/utils/utils.py", line 51, in save_hf_format os.makedirs(output_dir) File "/usr/lib/python3.8/os.py", line 223, in makedirs mkdir(name, mode) FileExistsError: [Errno 17] File exists:...

s-isaev

Missing key(s) in state_dict for bias in attention blocks

1

I am trying to run step 3 of the RLHF examples using a RewardModel checkpoint that I trained using step 2 of the examples. For every step, I used the...

EikeKohl

When running Stage-3 scripts with enable_hybrid_engine encountered errors

5

I was using script from step3_rlhf_finetuning/training_scripts/single_node/run_6.7b.sh, I met some errors. I used 7B Llama models as actor and critic respectively and set enable_hybrid_engine argument, I got errors like below: │...

DwarfWarriors

Stage-3 code about ppo loss may have something wrong

1

In `applications/DeepSpeed-Chat/training/step3_rlhf_finetuning/main.py`, `critic_loss` and `actor_loss` are strangely added to each other. I am so confused about it.

LuciusMos

single gpu 6.7b lora CUDA OOM with A6000 48G

4

I am trying to run DeepSpeed-Chat Example with single gpu, Nvidia A6000 48G. I could run all 3 steps well using 1.3b example. But when I run `single_gpu/run_6.7b_lora.sh`, I got...

HyeongminMoon

DeepSpeed Op Builder: Installed CUDA version 12.0 does not match the version torch was compiled with 11.8

1

**this is the error from training.log:** > Traceback (most recent call last): File "/data/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py", line 339, in main() File "/data/DeepSpeedExamples/applications/DeepSpeed-Chat/training/step1_supervised_finetuning/main.py", line 271, in main optimizer = AdamOptimizer(optimizer_grouped_parameters, File "/home/ps/anaconda3/envs/pt/lib/python3.10/site-packages/deepspeed/ops/adam/fused_adam.py", line...

yudonglee