optillm icon indicating copy to clipboard operation
optillm copied to clipboard

deepseek can't support n > 1

Open femto opened this issue 1 year ago • 3 comments

Hello, while trying bon, deepseek reports error since deepseek can't support n > 1, so I added

    except:
        for _ in range(n):
            response = client.chat.completions.create(
                model=model,
                messages=messages,
                max_tokens=4096,
                n=1,
                temperature=1
            )
            completions.extend([choice.message.content for choice in response.choices])
            bon_completion_tokens += response.usage.completion_tokens

femto avatar Nov 03 '24 09:11 femto

This is good, can you also update it for moa, mcts and pvg approaches? It should then fix #67 .

codelion avatar Nov 03 '24 14:11 codelion

Another request for something like this with some discussion is here #99

codelion avatar Nov 18 '24 22:11 codelion

CLA assistant check
All committers have signed the CLA.

CLAassistant avatar Dec 14 '24 00:12 CLAassistant