dspy
dspy copied to clipboard
[Bug] Together AI completions bugs
Hi, I believe there are a few bugs in the together API. When hitting the chat/completions endpoint, the response json I get doesn't contain an "output" field, but the Together client requires one.
changing
resp_json["output"].get("choices", [])[0]
to
resp_json.get("choices", [])[0]
fixes the issue. (and should be done both when the chat api is being used and when it isn't)
In addition, when not using the chat api, the url is by default set to just api_base, but it should probably be set to the completions endpoint like
url = f"{self.api_base}/completions"
so that when api_base is "https://api.together.xyz/v1", the natural choice, it would become "https://api.together.xyz/v1/completions", which is the correct endpoint.
These issues are still present as of 5/13.