cloudflare-docs icon indicating copy to clipboard operation
cloudflare-docs copied to clipboard

Chat completions for ai worker models is broken

Open kittykatattack opened this issue 1 year ago • 1 comments

Existing documentation URL(s)

https://developers.cloudflare.com/ai-gateway/providers/workersai/

What changes are you suggesting?

The code that shows how access an Open AI compatible chat completion endpoint for ai workers is incorrect:

curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/workers-ai/v1/chat/completions \
 --header 'Authorization: Bearer {cf_api_token}' \
 --header 'Content-Type: application/json' \
 --data '{
      "model": "@cf/meta/llama-3-8b-instruct",
      "messages": [
        {
          "role": "user",
          "content": "What is Cloudflare?"
        }
      ]
    }
'

This consistently produces the following:

{"result":null,"success":false,"errors":[{"code":7009,"message":"Upstream service unavailable"}],"messages":[]}

Additional information

This works fine without issue using the same credentials:

curl https://gateway.ai.cloudflare.com/v1/{account_id}/{gateway_slug}/workers-ai/@cf/meta/llama-3-8b-instruct \
 --header 'Authorization: Bearer {cf_api_token}' \
 --header 'Content-Type: application/json' \
 --data '{"prompt": "What is Cloudflare?"}'

kittykatattack avatar Jun 16 '24 12:06 kittykatattack