Qwen2.5 icon indicating copy to clipboard operation
Qwen2.5 copied to clipboard

Inference issue about qwen2-72b-instruct

Open TuTuHuss opened this issue 5 months ago • 4 comments

Hi there, I am attempting to utilize the qwen2-72b-instruct (API) for a continuation text task; however, the output lacks punctuation despite my inclusion of the directive "you should add necessary punctuations in the result" within the system_prompt. Could you please advise on how to rectify this issue?

class qwen():
    def __init__(self):
        self.client = OpenAI(
            api_key="",
            base_url="https://dashscope.aliyuncs.com/compatible-mode/v1"
        )

    def call(self,system_prompt:str,user_prompt:str) -> str:
        response = self.client.chat.completions.create(
            model="qwen2-72b-instruct",
            messages = [
                {'role': 'system', 'content': system_prompt},
                {'role': 'user', 'content': user_prompt}
            ],
            temperature=0.7,
            top_p=1,
            max_tokens=256
        )
        return response.choices[0].message.content

TuTuHuss avatar Sep 10 '24 06:09 TuTuHuss