DeepSpeed-Chat Step-3 tensorboard loss figures with multiple training epochs

[GeekDream-x]

Hi, when training RLHF step-3, I set parameters related to epochs as:

• ppo_epochs = 1
• num_train_epochs = 30

and I found that the numbers of lines in "actor_loss", "actor_loss_sum", "critic_loss", "critic_loss_sum" and "reward" (saved in the folder "step3_tensorboard_logs") are the same as the parameter "num_train_epochs", like :

The question is:

How can I make them display as the figure for "train_loss" or "lr" (saved in the folder "ds_tensorboard_logs") like the below one, where the line for the whole 30 epochs as a whole ( a single line):

Thanks!