dspy icon indicating copy to clipboard operation
dspy copied to clipboard

[Bug] Together AI completions bugs

Open adam-simple opened this issue 1 year ago • 1 comments
trafficstars

Hi, I believe there are a few bugs in the together API. When hitting the chat/completions endpoint, the response json I get doesn't contain an "output" field, but the Together client requires one.

changing resp_json["output"].get("choices", [])[0] to resp_json.get("choices", [])[0] fixes the issue. (and should be done both when the chat api is being used and when it isn't)

In addition, when not using the chat api, the url is by default set to just api_base, but it should probably be set to the completions endpoint like url = f"{self.api_base}/completions" so that when api_base is "https://api.together.xyz/v1", the natural choice, it would become "https://api.together.xyz/v1/completions", which is the correct endpoint.

adam-simple avatar Mar 11 '24 02:03 adam-simple

These issues are still present as of 5/13.

isaacbmiller avatar May 13 '24 20:05 isaacbmiller