genkit icon indicating copy to clipboard operation
genkit copied to clipboard

[Feature Request] Simple mechanism for setting/accessing ALS from flows and tools

Open mbleigh opened this issue 1 year ago • 0 comments

When building tool plugins, it's useful for the tools to be as broadly defined as possible; however, when actually using those tools it's useful for the LLM to have as narrow and concise a definition as possible.

It would be great to be able to do something like:

const myTool = defineTool({name: "myTool"}, async input => {
  const client = new WikiClient(getContext("wiki.server"));
});

defineFlow({name: "myFlow"}, async () => {
  setContext("wiki.server", {...});
  return generate({tools: [myTool], ...}); // passes context via ALS
});

The reason this is different from plugin config is that I discovered there are basically three kinds of config for tools:

  1. Initialization-time config (plugin config handles this)
  2. Inference-time config (LLM provides these arguments)
  3. Contextual config (e.g. Auth, or in this case wiki server config)

It would be great to have facilities for (3).

mbleigh avatar May 21 '24 06:05 mbleigh