Ke Meng
Ke Meng
PR #815 fixes #740.
@jgong5 Thank you for your reply. I believe the CPU has the potential to replace the GPU as a cost-efficient solution for LLM inference. Although the GPU is fast, it...
solved by https://github.com/alibaba/GraphScope/pull/2130
> For CPU (IPEX), this growth TREND appears abnormal when the batch size > 8, and it even decreases. Increasing the batch size actually becomes less efficient, which I think...