DeepSpeedExamples
DeepSpeedExamples copied to clipboard
Why does the chat.py script model answer normally, but answer repeatedly when step3
I deployed the model using the chat.py script and the model answered normally, but the output of the actor model was repeated throughout the step3.
chat.py:
step3:
actor model: llama2-13b rw model: llama2-13b hybrid engin enable