When running Stage-3 scripts with enable_hybrid_engine encountered errors

[DwarfWarriors]

I was using script from step3_rlhf_finetuning/training_scripts/single_node/run_6.7b.sh, I met some errors. I used 7B Llama models as actor and critic respectively and set enable_hybrid_engine argument, I got errors like below:

│ /root/miniconda3/envs/coati/lib/python3.10/site-packages/deepspeed/runtime/hybrid_engine.py:398 │ │ in step │ │ │ │ 395 │ │ │ 396 │ def step(self, lr_kwargs=None): │ │ 397 │ │ super().step(lr_kwargs=lr_kwargs) │ │ ❱ 398 │ │ if(self._inference_containers[0].module.attention.attn_qkvw is not None and \ │ │ 399 │ │ │ self._inference_containers[0].q_k_v is not None): │ │ 400 │ │ │ for inference_container in self._inference_containers: │ │ 401 │ │ │ │ inference_container.reset_qkv() │ ╰──────────────────────────────────────────────────────────────────────────────────────────────────╯ IndexError: list index out of range

How can I solve this issue? Thx : )

[YaguangGong]

same error when using GLM as actor.

[l294265421]

same error using LLaMA as an actor when zero stage = 0

[yaozhewei]

Hi, Llama is not supported yet. Please stay tuned :)

[aimetrics]

Looking forward!

[l294265421]

same error using LLaMA as an actor when zero stage = 0

When zero stage = 0, removing enable_hybrid_engine option works.

[qinqinqaq]

Looking forward!

[mcc311]

Looking forward!

[alphanlp]

same error using LLaMA as an actor when zero stage = 3

[alphanlp]

same error using LLaMA as an actor when zero stage = 0

When zero stage = 0, removing enable_hybrid_engine option works.

i just try this way, but i find the training process is very slow!