griptape icon indicating copy to clipboard operation
griptape copied to clipboard

Use underlying SDKs to automatically fill in model options when creating drivers

Open shhlife opened this issue 11 months ago • 2 comments

Is your feature request related to a problem? Please describe. Customers can have a difficult time knowing what models to choose when creating drivers if they don't already know what models exist. For things like gpt-4o it's pretty easy to type.. but for options like sonnet unless you know the exact ids, it's a challenge.

It would be wonderful to be able to use the underlying model SDKs to grab the list of models available and dynamically fill them.

Describe the solution you'd like Here's a loom demonstrating a modification I did with Anthropic to pull from their list: https://www.loom.com/share/d7299383d6894101bc14db3fe8c05e43?sid=d4fedfee-ec64-45fa-9477-9a0a12dac9db

And to do the same with OpenAiChatPromptDriver I modified openai_chat_prompt_driver.py:

in the if TYPE_CHECKING: section:

if TYPE_CHECKING:
    from collections.abc import Iterator

    from openai.types.chat.chat_completion_chunk import ChoiceDelta
    from openai.types.chat.chat_completion_message import ChatCompletionMessage
    from openai.types.chat_model import ChatModel # Pull from OpenAi's types
    from griptape.tools import BaseTool

and then I switched the model param:

    # model: str = field(kw_only=True, metadata={"serializable": True})
    model: ChatModel = field(kw_only=True, metadata={"serializable": True}) # Use OpenAi's ChatModel

Describe alternatives you've considered We could manually create the list, but this seems way more efficient and will adjust automatically as the libraries change.

shhlife avatar Dec 13 '24 17:12 shhlife

I've been slowly implementing this in the Griptape ComfyUI nodes.. if it's helpful, here are the types for OpenAi:

from openai.types import AudioModel, ChatModel, EmbeddingModel, ImageModel
from openai.types.audio.speech_model import SpeechModel

shhlife avatar Dec 20 '24 23:12 shhlife

Blocked on #1587. Need to move imports out of TYPE_CHECKING to accomplish this.

collindutter avatar Jan 21 '25 22:01 collindutter