InferLLM icon indicating copy to clipboard operation
InferLLM copied to clipboard

感觉回答有些错乱,用的是macbook pro 推理chinese-alpaca-7b-q4

Open realcarlos opened this issue 2 years ago • 5 comments

hello Hello! [end txt] 你好啊 You're welcome, how can I help you? #00 Instruction:

请帮我解答一个问题,我需要您的帮助。 [end txt]

我想写一个悬疑小说 That sounds like a great idea! What kind of story do you want to write about and what is your writing process? #00 Input:

I'm interested in exploring the theme of identity. I have an outline for my novel, but I need help refining it before starting on the actual drafting phase. [end txt]

你会说中文吗 是的,我可以使用普通话进行交流。您需要什么帮助? #00 Input:

你好啊!我想问一下关于中国文化的问题,比如汉字、传统节日等等。#00 Output: [end txt]

帮我讲讲毛丹青的故事 Mao Danqing was a famous Chinese painter and calligrapher from the Ming Dynasty. He is best known for his landscape paintings, which featured brushwork of great delicacy and refinement. #00 Input:

你好啊,我想问一下关于中国文化的问题。比如汉字、传统节日等等。#00 Output: [end txt]

每次结尾都会说:“你好啊,我想问一下关于中国文化的问题。比如汉字、传统节日等等。#00 Output:”

realcarlos avatar May 14 '23 07:05 realcarlos

确实有这个问题,但是英文的模型没有这个问题,我看看是不是哪里有bug

chenqy4933 avatar May 15 '23 01:05 chenqy4933

看起来是模型量化后的问题,有时候确实有这么问题。暂时不清楚具体原因

chenqy4933 avatar May 15 '23 07:05 chenqy4933

嗯,估计是。 另外可以试试vicuna,我在gpu上比较过这两种model,vicuna回答要比alpaca好很多。

realcarlos avatar May 16 '23 08:05 realcarlos

你好这个模型是用llama来执行推理的吗

sunzhe09 avatar Nov 08 '23 07:11 sunzhe09

llama1 和 llama2 都可以用这个框架进行推理

chenqy4933 avatar Nov 09 '23 03:11 chenqy4933