optillm
optillm copied to clipboard
deepseek can't support n > 1
Hello, while trying bon, deepseek reports error since deepseek can't support n > 1, so I added
except:
for _ in range(n):
response = client.chat.completions.create(
model=model,
messages=messages,
max_tokens=4096,
n=1,
temperature=1
)
completions.extend([choice.message.content for choice in response.choices])
bon_completion_tokens += response.usage.completion_tokens
This is good, can you also update it for moa, mcts and pvg approaches? It should then fix #67 .
Another request for something like this with some discussion is here #99