griptape
griptape copied to clipboard
Create a Groq driver
- [X] I have read and agree to the contributing guidelines.
Is your feature request related to a problem? Please describe. Groq is very fast, and is somewhat OpenAI API compatible.
Using something like this:
import os
from dotenv import load_dotenv
from griptape.drivers import OpenAiChatPromptDriver
from griptape.structures import Agent
from griptape.utils import Chat
from rich import print as print
load_dotenv()
GROQ_API_KEY = os.getenv("GROQ_API_KEY")
agent = Agent(
prompt_driver=OpenAiChatPromptDriver(
api_key=GROQ_API_KEY,
base_url="https://api.groq.com/openai/v1",
model="llama3-groq-70b-8192-tool-use-preview",
),
stream=True,
)
Chat(agent).start()
is extremely fast - however, if you give it a tool you get this response:
WARNING:root:<RetryCallState 2380201637520: attempt #1; slept for 0.0; last result: failed (ValueError Invalid JSON input for ToolAction)
Here's the OpenAI API compatibility doc: https://console.groq.com/docs/openai and tools: https://console.groq.com/docs/tool-use
it says it should work with llama3-groq-70b-8192-tool-use-preview, but it doesn't appear to.