SWE-agent icon indicating copy to clipboard operation
SWE-agent copied to clipboard

Minimal changes to support models of Groq Cloud

Open crjaensch opened this issue 10 months ago • 1 comments

Reference Issues/PRs

What does this implement/fix? Explain your changes.

GroqCloud offers one of the fastest inference engines available today with 500-700 tokens/sec. They host well-known OSS models, such as Mixtral 8x7B, and support an API that is 100% compatible with the OpenAI API. I made minimal changes to model.py to support calling that API.

In addition, I created two files that show how to call the CLI for both GroqCloud models as well as local Ollama models.

Any other comments?

🧡 Thanks for contributing!

crjaensch avatar Apr 08 '24 06:04 crjaensch

PR with same goal: #108

klieret avatar Apr 08 '24 20:04 klieret