dspy icon indicating copy to clipboard operation
dspy copied to clipboard

feat: adds Cloudflare Workers AI

Open isbecker opened this issue 9 months ago • 2 comments

This feature adds support for using Cloudflare Workers AI. Using Cloudflare gives support for a wide variety of open source models.

env variables

CLOUDFLARE_ACCOUNT_ID CLOUDFLARE_API_KEY

lm = dspy.CloudflareAI(model="@cf/meta/llama-3-8b-instruct", 
          account_id="", # optional, will check env
          api_key="" # optional, will check env
)
dspy.settings.configure(lm=lm)
# ...use normally...

isbecker avatar May 01 '24 13:05 isbecker

Hi @isbecker , thanks for the PR!

Could you also add documentation for the CloudfareAI LM to provide context as done for the LMs in our documentation here??

Also, could you please rebase and merge this branch with the latest version on main? Seems like there was a failing test on our end (nothing wrong with this PR) that just needs an update. Ready to merge after that!

arnavsinghvi11 avatar May 04 '24 23:05 arnavsinghvi11

Will do both, I'm away from computer today, but I will have time tomorrow to make those changes 🙂

isbecker avatar May 05 '24 01:05 isbecker

@arnavsinghvi11 looks good?

  • [x] tests pass locally
  • [x] docs added

isbecker avatar May 10 '24 13:05 isbecker

Thanks @isbecker !

arnavsinghvi11 avatar May 11 '24 18:05 arnavsinghvi11