Enable GPT functions

[ihh]

Mentioned by @jxnl and @cmungall in #113. Actually this is a lot simpler/lighter than what is discussed there and in @kmad's PR #66, but it does what I need it to, and wraps the OpenAI API pretty simply I think

[simonw]

Initially I had thought that this wasn't the right direction to go - that function call support should be integrated more deeply into the library rather than just being provided by options like this.

But when I started thinking through that deeper integration, I realized that using options as a starting point was likely the first step on getting this to work!

I don't think the path-to-a-file thing is quite the right default though. I just tried switching the option to accepting JSON directly and got this example to work:

llm -o functions '[
    {
        "name": "extract_locations",
        "description": "Extract all locations mentioned in the text",
        "parameters": {
            "type": "object",
            "properties": {
                "locations": {
                    "type": "array",
                    "items": {
                        "type": "object",
                        "properties": {
                            "name": {
                                "type": "string",
                                "description": "The name of the location"
                            },
                            "country_iso_alpha2": {
                                "type": "string",
                                "description": "The ISO alpha-2 code of the country where the location is situated"
                            }
                        },
                        "required": [
                            "name",
                            "country_iso_alpha2"
                        ]
                    }
                }
            },
            "required": [
                "locations"
            ]
        }
    }
]' 'I went to London and Utrecht last week'

Output:

{"function_call": {
"name": "extract_locations",
"arguments": {
  "locations": [
    {
      "name": "London",
      "country_iso_alpha2": "GB"
    },
    {
      "name": "Utrecht",
      "country_iso_alpha2": "NL"
    }
  ]
}
}}

If you want to use a path, you can do this instead:

llm -o functions "$(cat functions.json)" 'I went to London and Utrecht last week'

I'm going to land this as a starting point and continue to iterate on it (though it's going to be a bit tricky to fix the merge conflicts with the changes I made in #284)

[jxnl]

what are your thoughts on being able to enable a path to a pydantic object?

where

# models.py
class ExtractLocation(BaseModel): 
    ...

llm -o model models.ExtractLocation 'I went to London and Utrecht last week'

{
  "locations": [
    {
      "name": "London",
      "country_iso_alpha2": "GB"
    },
    {
      "name": "Utrecht",
      "country_iso_alpha2": "NL"
    }
  ]
}

Theres a few workflows where i imagine i'd want to be able to

cat *.txt | llm -o model models.ExtractLocation > data.jsonl

[simonw]

That's the direction I was initially thinking about going in, since LLM has Pydantic as a dependency already.

I'm still trying to flesh out all of the different ways this could be used. The thing I'm personally most interested in is being able to add a docstring and some type annotations to a Python function and have the functions schema for that generated automatically - but defining a Pydantic class (or just a regular Python dataclass) for the extract-structured-data case is interesting too.

[simonw]

Part of the design challenge here is that I want to address all of these:

• The CLI use-case
• The interactive chat use-case (llm chat but also the web chat UI I'm putting together)
• The Python library use-case

Most importantly, I want to be able to support LLM plugins that provide new functions which can be executed using OpenAI functions - and eventually through imitations of OpenAI functions that work against Llama 2 etc.

[jxnl]

Yeah, I think the advantages of pydantic really come in if you have more complex nested structures. I have some examples on my docs, but they would probably be too complicated to define in the command line.

Often it's noy extract one object. But yield many objects from the text.