Autogen Tool, ToolFunction, Parameters, Property schema when opt-in to using type-hinted function params

[anthonywu]

😩 Problem

In the API docs for passing in tools= spec sequence to the chat model, the docs suggest developers hand-craft the Tool, ToolFunction, Parameters, Property schema'ed docs to the tools= arg. This is very repetitive.

For example, building up this JSON is laborious, given we already have defined the tool calling function.

{
      "type": "function",
      "function": {
        "name": "get_current_weather",
        "description": "Get the current weather for a location",
        "parameters": { 😩
          "type": "object",
          "properties": {
            "location": {
              "type": "string",
              "description": "The location to get the weather for, e.g. San Francisco, CA"
            }, 😩
            "format": {
              "type": "string",
              "description": "The format to return the weather in, e.g. 'celsius' or 'fahrenheit'",
              "enum": ["celsius", "fahrenheit"] 😩
            }
          },
          "required": ["location", "format"] 😩
        }
      }

Improvement Proposal

I propose we can solve it by giving users an option to type hint their functions and get back an automatic schema.

The scope is:

• must only use Python standard library and native type typing helpers
• your literal function name in .py is the function name in the spec, no need to type it twice
• function docstring is the Function description, this should be a wide-spread developer habit, some possibility to inject prompts here
• parameter description provided by the second arg of Annotated special typing form
• required/optional state is hinted by Optional annotation or classic keyword args with default values
• the documented types are supported
  • bool -> "boolean"
  • float -> "number"
  • int -> "integer"
  • str -> "string"
  • Sequence -> "array"

The user expectation is:

• this is optional, if you want to hand craft the spec, go for it
• the generated spec is just plain JSON, you can edit it however you wish if the auto generated spec is not exactly what you need

Example

@ollama.annotated_tool
def some_tool_function(
    expr_1: t.Annotated[str, "first simple str value"],
    expr_2: t.Annotated[str, "second simple str value"],
    req_int_arg: t.Annotated[int, "a required integer"],
    req_float_arg: t.Annotated[float, "any real number"],
    req_list_arg_1: t.Annotated[list, "any builtin list"],
    req_list_arg_2: t.Annotated[t.List[t.Any], "any typed List"],
    req_enum_1: t.Annotated[ExampleEnum, "required foo bar or baz"],
    opt_arg_1: t.Annotated[t.Optional[str], "an optional string"] = None,
    opt_enum_1: t.Annotated[t.Optional[ExampleEnum], "optional foo bar or baz"] = None,
    opt_enum_2: ExampleEnum = ExampleEnum.FOO,
    opt_builtin_str_1: str = "foobar",
    opt_builtin_int_1: int = 1e6,
    opt_builtin_float_1: float = 1.0,
    opt_builtin_list_1: list = None,
):
    """a test case for Annotating a tool function's parameters.
    the docstring of the function is the Function.description."""
    return None

some_tool_function.tool_schema would provide this, usable as an item in tools= list

{
  "type": "function",
  "function": {
    "name": "example_1",
    "description": "a test case for Annotating a tool function's parameters.\n    the docstring of the function is the Function.description.",
    "parameters": {
      "type": "object",
      "required": [
        "expr_2",
        "expr_1",
        "req_int_arg",
        "req_list_arg_2",
        "req_enum_1",
        "req_float_arg",
        "req_list_arg_1"
      ],
      "properties": {
        "expr_1": {
          "type": "string",
          "description": "first simple str value",
          "is_optional": false,
          "enum": null
        },
        "expr_2": {
          "type": "string",
          "description": "second simple str value",
          "is_optional": false,
          "enum": null
        },
        "req_int_arg": {
          "type": "integer",
          "description": "a required integer",
          "is_optional": false,
          "enum": null
        },
        "req_float_arg": {
          "type": "number",
          "description": "any real number",
          "is_optional": false,
          "enum": null
        },
        "req_list_arg_1": {
          "type": "array",
          "description": "any builtin list",
          "is_optional": false,
          "enum": null
        },
        "req_list_arg_2": {
          "type": "array",
          "description": "any typed List",
          "is_optional": false,
          "enum": null
        },
        "req_enum_1": {
          "type": "string",
          "description": "required foo bar or baz",
          "is_optional": false,
          "enum": [
            "foo",
            "bar",
            "baz"
          ]
        },
        "opt_arg_1": {
          "type": "string",
          "description": "an optional string",
          "is_optional": true,
          "enum": null
        },
        "opt_enum_1": {
          "type": "string",
          "description": "optional foo bar or baz",
          "is_optional": true,
          "enum": [
            "foo",
            "bar",
            "baz"
          ]
        },
        "opt_enum_2": {
          "type": "string",
          "description": "opt enum 2",
          "is_optional": true,
          "enum": [
            "foo",
            "bar",
            "baz"
          ]
        },
        "opt_builtin_str_1": {
          "type": "string",
          "description": "opt builtin str 1",
          "is_optional": true,
          "enum": null
        },
        "opt_builtin_int_1": {
          "type": "integer",
          "description": "opt builtin int 1",
          "is_optional": true,
          "enum": null
        },
        "opt_builtin_float_1": {
          "type": "number",
          "description": "opt builtin float 1",
          "is_optional": true,
          "enum": null
        },
        "opt_builtin_list_1": {
          "type": "array",
          "description": "opt builtin list 1",
          "is_optional": true,
          "enum": null
        }
      }
    }
  }
}

Out of Scope

Not attempting to scope creep to:

• consistency with OpenAI Python SDK's similar implementation
• not using pydantic so to not obligate pip install ollama users to also install pydantic (see comment in https://github.com/ollama/ollama-python/pull/234#issuecomment-2253032468)
• make this a stand alone pip installable library, I think the implementation is deeply coupled to ollama's _types.py implementation. I am open to doing this implementation as a separate library if the maintainers believe this syntactic sugar is out of scope.

Related

• #234 "Add Tool Registry Feature" is highly related, however it was closed for recommendation to package as a third party library. My implementation here adds no additional dependencies and remains entirely optional.
• There was a related attempt to solve this problem: https://github.com/robocorp/llmfoo but that attempt sends the source of the function to OpenAI API to let the LLM generate the tool schema. My solution here is entirely local and relies on the Python 3 standard type utils only, 100% type reflection.