nebuly icon indicating copy to clipboard operation
nebuly copied to clipboard

Published 20 hours ago • verified publisher icon nebuly-ai

[Chatllama] assert _MODEL_PARALLEL_GROUP is not None, "model parallel group is not initialized"

Open seasidemym opened this issue 1 year ago • 3 comments

How to use parallel training in actor. Simple demo is not work. The project is not solid.

seasidemym avatar Mar 08 '23 10:03 seasidemym

Hi @seasidemym, thank you so much for reaching out. Today, we are going to release a more stable version of ChatLLaMA, hopefully you will enjoy it more. Please let us know if you have any other feedback 😄

diegofiori avatar Mar 08 '23 10:03 diegofiori

Any update? I encountered same issue. I simply set environment variables by two command: export MP=1, export WORLD_SIZE=1 . Then start training of actor with "fairscale True" in config.yaml.

tqjack avatar Mar 10 '23 01:03 tqjack

Hi @seasidemym @tqjack
This issue should have been fixed in the previous release in which was missing the model parallel initialisation.

PierpaoloSorbellini avatar Apr 03 '23 14:04 PierpaoloSorbellini