ZHENG XIN

Some trouble when running your code

9

“Traceback (most recent call last): File "train.py", line 332, in main() File "train.py", line 324, in main train_model(model, data, optim, i, params) File "train.py", line 179, in train_model score =...

questions about the result of abductive

1

Given the input `Ray drive his car on a steep mountain road. | Ray was fine but his car was totaled.` the ranked hypothesis is `The car is car, The...

https://github.com/microsoft/DeepSpeed/blob/4e886f0568832d292183926bcc1a9105def25f2c/deepspeed/autotuning/autotuner.py#L714-L726 It is possible that fast_best_metric_val could be None at fast mode tuning if all experiments are OOM, while "slow" mode gets some results. So this line could go wrong:...

https://github.com/microsoft/DeepSpeed/blob/4e886f0568832d292183926bcc1a9105def25f2c/deepspeed/autotuning/autotuner.py#L433-L434 But this cannot keep the user's configuration of zero stage scope. Shall Line 434 be deleted? https://github.com/microsoft/DeepSpeed/pull/3473 Or are there other concerns?

About DEFAULT_MIN_MEM_CONFIG in autoning

5

https://github.com/microsoft/DeepSpeed/blob/b361c72761d97f5a1714a3e91d1f7c36fd3cfdd8/deepspeed/autotuning/constants.py#L142-L148 That doesn't include parameter and optimizer offload, which is memory minimal. ``` "offload_optimizer": { "device": "cpu", "pin_memory": true }, "offload_param": { "device": "cpu", "pin_memory": true }, ```

Support llama3 fine-tune

6

## Why are these changes needed? Support llama3 fine-tune, which is the extension of https://github.com/lm-sys/FastChat/pull/3259. Also, the length-1 tokenization mismatch is fixed. ## Related issue number (if applicable) ## Checks...