张洛阳 comments

Repositories
Issues
Comments

Results 1 comments of


                                            张洛阳

Multi-gpu training example?

By this way, qlora can use deepspeed. I have two gpu devices. local_rank = int(os.environ.get("LOCAL_RANK", 0)) if local_rank ==0: max_memory = {0 : "{0}MB".format(45 * 1024), 1: "0MB"} else: max_memory...