workerd icon indicating copy to clipboard operation
workerd copied to clipboard

Python worker can't access env.AI methods like run after binding

Open neeravkumar opened this issue 1 year ago • 1 comments

Can't create a python worker that can access the AI bindings

from js import Response

async def on_fetch(request, env):
    simple = "{'prompt': 'Tell me a random quote'}"
    response = await env.AI.run("@cf/meta/llama-2-7b-chat-int8", simple)
    return Response.new(response)

returns

Traceback (most recent call last):
  File \"/session/metadata/entry.py\", line 5, in on_fetch
    response = await env.AI.run(\"@cf/meta/llama-2-7b-chat-int8\", simple)
                     ^^^^^^^^^^
AttributeError: run

neeravkumar avatar Apr 08 '24 20:04 neeravkumar

For context, in JavaScript you would write:

import { Ai } from '@cloudflare/ai'

export interface Env {
  // If you set another name in wrangler.toml as the value for 'binding',
  // replace "AI" with the variable name you defined.
  AI: any;
}

async function fetch(request: Request, env: Env) {
  const ai = new Ai(env.AI);
  const response = await ai.run('@cf/meta/llama-2-7b-chat-int8', {
      prompt: "What is the origin of the phrase Hello, World"
    }
  );
}

So the run method is on the Ai wrapper and not on env.AI itself. We need some plan for how to allow Python workers to access equivalent functionality.

hoodmane avatar Apr 08 '24 20:04 hoodmane