deadlykitten4
deadlykitten4
Hi, do you understand what does the train/8 mean? I am also confused about this.
I used `pip install vllm` to install. And I found that if run script in the directory vllm, then it won't show this error. I don't know the reason why...
Hi, I am still confused about the evaluation of the throughput, cause the result I got (as you can see in the picture) is quite different from the result in...
@ldengjie Nope, I still can't figure out what is the problem of it. And most of my results show that there is no speedup compared to the original model. And...
Hi, @dellixx, may I ask how you run this code on LLaMA3? Cause I have upgraded the transformers version and modified the SVD_LlamaAttention class, but I obtained an extremely bad...
你用deepspeed了吗，貌似在你提供的脚本里没有看到，加上deepspeed试试
Very useful solution!! Thanks @pennyLuo-hub
I want to evaluate the ppl (perplexity) of LLaDA. But the results are extremely bad based on my implementation. I think there may be some issues I didn't figure out....
Okay, thanks for your help!