genkit
genkit copied to clipboard
[Feature Request] Simple mechanism for setting/accessing ALS from flows and tools
When building tool plugins, it's useful for the tools to be as broadly defined as possible; however, when actually using those tools it's useful for the LLM to have as narrow and concise a definition as possible.
It would be great to be able to do something like:
const myTool = defineTool({name: "myTool"}, async input => {
const client = new WikiClient(getContext("wiki.server"));
});
defineFlow({name: "myFlow"}, async () => {
setContext("wiki.server", {...});
return generate({tools: [myTool], ...}); // passes context via ALS
});
The reason this is different from plugin config is that I discovered there are basically three kinds of config for tools:
- Initialization-time config (plugin config handles this)
- Inference-time config (LLM provides these arguments)
- Contextual config (e.g. Auth, or in this case wiki server config)
It would be great to have facilities for (3).