Ke Meng

Results 14 comments of Ke Meng

@jgong5 Thank you for your reply. I believe the CPU has the potential to replace the GPU as a cost-efficient solution for LLM inference. Although the GPU is fast, it...

solved by https://github.com/alibaba/GraphScope/pull/2130

> For CPU (IPEX), this growth TREND appears abnormal when the batch size > 8, and it even decreases. Increasing the batch size actually becomes less efficient, which I think...