DeepSpeedExamples icon indicating copy to clipboard operation
DeepSpeedExamples copied to clipboard

Published 20 hours ago • verified publisher icon microsoft

【BUG】occur error：AttributerError：'DeepSpeedHybridEngine' object has no attribute 'mp_group' whiling run llama7b for step3/rlhf/ppo

Open Pattaro opened this issue 1 year ago • 3 comments

98DDB13F-60AE-4F7D-8979-9B287A2A4CC1

Pattaro avatar Apr 20 '23 15:04 Pattaro

Any solutions? I encountered the same issue with bloomz model, so far I just removed --enable_hybrid_engine to bypass the issue, and the program runs. However, guess the efficiency would reduce.

Chevolier avatar Apr 21 '23 02:04 Chevolier

It seems enable_hybrid_engine only support for OPT models.

DwarfWarriors avatar Apr 21 '23 03:04 DwarfWarriors

try adjusting the --inference_tp_size to a lower number, it may be you don't have enough GPUs across your nodes.

[bug]AttributeError: 'DeepSpeedHybridEngine' object has no attribute 'mp_group' #525

kkk935208447 avatar Mar 18 '24 01:03 kkk935208447