Qwen2.5
Qwen2.5 copied to clipboard
Inference issue about qwen2-72b-instruct
Hi there, I am attempting to utilize the qwen2-72b-instruct (API) for a continuation text task; however, the output lacks punctuation despite my inclusion of the directive "you should add necessary punctuations in the result" within the system_prompt. Could you please advise on how to rectify this issue?
class qwen():
def __init__(self):
self.client = OpenAI(
api_key="",
base_url="https://dashscope.aliyuncs.com/compatible-mode/v1"
)
def call(self,system_prompt:str,user_prompt:str) -> str:
response = self.client.chat.completions.create(
model="qwen2-72b-instruct",
messages = [
{'role': 'system', 'content': system_prompt},
{'role': 'user', 'content': user_prompt}
],
temperature=0.7,
top_p=1,
max_tokens=256
)
return response.choices[0].message.content