HUANG Fei

辛苦辛苦，现在把为什么需要这些额外信息的原因说的比较清楚啦

我们去年已经有同学整理过笔记 https://github.com/thunlp/OOP-THU/issues/47 如果只是重复课上内容可能贡献不大，大家可能也不太想看到重复内容。如果有新增部分，欢迎对之前的笔记做一定的补充。

请升级autogptq到0.7.0

@Modas-Li @magnificent1208 模型输出截断问题可能有各种不同原因（楼主应该就是没用fastchat导致的），如果有问题可以单开isseu或者在这个帖子下回复出问题的具体情况（比如模型大小，启动方式，模型输出，解码参数，命令行输出等）

I just find https://arxiv.org/pdf/1905.05471.pdf, who use method4 to evaluate conversational models. I don't know whether the paper is correct, but I met the problems when reproducing the results. However, it's...

I have seen your PR, but I don't think the problem in ``method5`` is fixed. It should average the numerators, not the probability. And there is also a problem in...

我们对比过，分数基本都是对的上的。如果输出差别很大可能是配置问题，麻烦提供具体的情况以便分析（包括模型、显卡环境、调用方法、有问题的输出）

It seems that PyTorch1.11 removes cub from the default including directory. **A direct workaround is using Pytorch1.10.** I am trying to include ``cub`` in pytorch1.11 and will update this issue...

I am trying to reproduce your environment... it may take some times before I can find a solution. If possible, you can also try using cuda>=11.0. Or just skip cuda...

@jchang98 I have pushed an update which manuanlly includes the cub library. Please re-clone this repo and try again.