LLMLingua icon indicating copy to clipboard operation
LLMLingua copied to clipboard

Which version of openai should be installed to reproduce gsm8k with llmlingua?

Open LYH-YF opened this issue 7 months ago • 3 comments

I installed the version 0.27.4 for runing the code examples/CoT.ipynb some error raised when running the following line

request_data = {
    "prompt": prompt,
    "max_tokens": 400,
    "temperature": 0,
    "top_p": 1,
    "n": 1,
    "stream": False,
    "stop": "\n\n",
}
response = openai.Completion.create(
    model="gpt-3.5-turbo-0301",
    **request_data,
)
print(json.dumps(response, indent=4))
openai.error.InvalidRequestErrorpython-BaseException
: This is a chat model and not supported in the v1/completions endpoint. Did you mean to use v1/chat/completions?

I update openai to version 0.28.1, the error also exists. Updating newer version doesn't work. So I change the code according to the error. It seems gpt-3.5-torbo-0301 only used for ChatCompletion.

request_data = {
    "messages": [{"role": "system", "content": ""}, {"role": "user", "content": prompt}],
    "max_tokens": 400,
    "temperature": 0,
    "top_p": 1,
    "n": 1,
    "stream": False,
    "stop": "\n\n",
}
response = openai.ChatCompletion.create(
    model="gpt-3.5-turbo-0301",
    **request_data,
)

The finally output is 0.439, far from 0.78+.

num_q 1319 correct 579 ratio 0.4390

Is there any suggestion for me?

  1. Should I use older openai or newer openai?
  2. Older openai may not support newer models, so I prefer to use newer openai (for more widely testing) to reproduce the result. But the result of gpt-3.5-torbo-0301 seems not good.

LYH-YF avatar Nov 17 '23 05:11 LYH-YF