binmakeswell comments

Results 359 comments of


                                            binmakeswell

[coati] How to get prompt_path and pretrain_dataset?

@gongel We have released it. Thanks. https://github.com/hpcaitech/ColossalAI/tree/main/applications/Chat#supervised-datasets-collection

[BUG]: AttributeError: 'LlamaForCausalLM' object has no attribute 'module'

Hi @mynewstart We have fixed it, please refer to #3475 and #3334 . Thanks.

[FEATURE]: Is hybrid parallelism supported in GPT demo?

Hi @nrailgun DP+ZERO is a simple way and enough for most cases. Examples of hybrid parallelism for large models will be provided soon. Thanks.

what is coati and how to install

> I also have this problem ,how do you solved? pip show it have already installed, but the train script appears this error ModuleNotFoundError: No module named 'coati' Hi @elven2016...

[BUG]: CUDA out of memory

Hi @janglichao @610lyn We have add How to train with limited resources. Thanks. https://github.com/hpcaitech/ColossalAI/tree/main/applications/Chat#faq

[BUG]: The LLama model trained in Lora mode is unable to perform normal reasoning

Hi @tianbuwei Thanks for the feedback, we are fixing LoRA bug. #3439

[BUG]: import error

Hi @yingtongxiong Sorry, we are currently in high-speed updates and the document may be a bit outdated. We suggest that you use images and language under example for testing. Thanks....

[BUG]: train_rm.py get lower acc!

Hi @Yutongamber Maybe it’s an inappropriate sh command. We have fixed it. Thanks. #3490 https://github.com/hpcaitech/ColossalAI/blob/main/applications/Chat/examples/train_rm.sh

[FEATURE]: add parts for unsupervised learning

Hi @wanghetongtt Yes, this can be done. However, this is currently beyond the scope of support from the open source community. Feel free to contact me via email [email protected] to...

[BUG]: RuntimeError(f"Parameter model.lm.parameters failed at the gradient reduction. " "Some unsupported torch function is operated upon this parameter.")

Hi @Youly172 Could you please provide more details to help us to reproduce it? e.g. What's example, command, env, and any changes?