flm模型和glm2模型输出结果不一致

[newsongwf]

[YerongLi]

glm1 也是不一致：

load from pf + merge + lora + cast to int4.

需要直接load int4的支持

Input : Who is Elon Musk?

Output:
...
m Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome
 proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasomeDjangoDjangoDjango

[newsongwf]

自带的prompt模版和glm2有差异，改成一样的以后有改善，但是还是有一些差异

[ztxz16]

目前提升了int4量化方法的精度，在原始模型上效果有较大提升，可以试试

[ZhouShaoyang]

glm1 也是不一致：

load from pf + merge + lora + cast to int4.

需要直接load int4的支持

Input : Who is Elon Musk?

Output:
...
m Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium Consortium proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome
 proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasome proteasomeDjangoDjangoDjango

@YerongLi ，你好，想了解一下merge lora后，转为int4 的fastllm模型，回答一直重复，这个情况有办法解决吗？现在遇到了同样的问题。原版模型是正常的，一旦merge lora后就变成这样了。