firebase
[FR] Support tool calling for any arbitrary (ollama?) model
Is your feature request related to a problem? Please describe. Currently several other frameworks support tool calling for ollama via a "wrapper", which I think has led this to become an expected feature. As of today, we rely on model api support for whether or not we can enable tool calling. We already retrofit functionality such as "context" and "system prompts" via middleware.
Describe the solution you'd like I have not fully thought this out, but we could add a middleware that would roughly do the following:
- Append instructions to the prompt that include tool definitions
- Append instructions to the prompt that specify the expected json output for a tool call request
- Translate json tool call requests into genkit tool call requests
- Tool calling should work as usual?
- Translate tool call responses back into "user" messages
Describe alternatives you've considered
- N/A
Additional context See also: https://github.com/firebase/genkit/issues/661
Notes: Such a feature is largely dependent on models that can reliably produce json output.
