rtp-llm icon indicating copy to clipboard operation
rtp-llm copied to clipboard
verified publisher icon alibaba

RTP-LLM 模式下，llama3.1 FP16 效果不一样

Open anigi98932 opened this issue 1 year ago • 1 comments

使用huggingface载入 llama3.1的生成结果，和RTP-LLM 载入结果不一致 同样的prompt，无法重现在RTP-LLM

anigi98932 avatar Sep 13 '24 01:09 anigi98932

对比一下input id和first token的hidden states?

ySingularity avatar Nov 18 '24 08:11 ySingularity