langchainjs icon indicating copy to clipboard operation
langchainjs copied to clipboard

Published 20 hours ago • verified publisher icon hwchase17

cloudflare[minor]: use native Cloudflare Workers AI binding

Open Cherry opened this issue 1 year ago • 5 comments

As of 2024-04-11, there's no need to use the @cloudflare/ai package and it's deprecated:

https://developers.cloudflare.com/workers-ai/changelog/

This swaps the code over to use the native binding directly, and updates @cloudflare/workers-types which now ships the Ai type. No other code changes are needed.

Cherry avatar May 05 '24 23:05 Cherry

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
langchainjs-docs ✅ Ready (Inspect) Visit Preview Sep 11, 2024 0:01am
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
langchainjs-api-refs ⬜️ Ignored (Inspect) Sep 11, 2024 0:01am

vercel[bot] avatar May 05 '24 23:05 vercel[bot]

Hey so just got wind that the Rest API methods will be available in the main SDK:

https://www.npmjs.com/package/cloudflare

We should wait to land this until that is generally ready

jacoblee93 avatar May 13 '24 21:05 jacoblee93

The general cloudflare SDK makes sense outside of Workers, but bindings inside of Workers are faster and more secure - no additional authentication required, so using it inside a Worker for anything that has native bindings doesn't make a lot of sense in my experience.

Updating https://github.com/Cherry/langchainjs/blob/4b9192176440b4acab9eac60a140200640d8bee2/libs/langchain-cloudflare/src/chat_models.ts#L78 to use the official SDK once the AI endpoints are available there makes sense, but https://github.com/Cherry/langchainjs/blob/fix/workers-ai-native-binding/libs/langchain-cloudflare/src/embeddings.ts seems to be intended for use only within a Worker.

Cherry avatar May 13 '24 21:05 Cherry

Yeah for sure but ideally we don't lose functionality for people using it outside workers - I've done that myself in demos.

It should accept either a binding or a config object/client IMO.

jacoblee93 avatar May 13 '24 22:05 jacoblee93

Agreed that would be ideal, but it doesn't look like it does today: https://github.com/langchain-ai/langchainjs/blob/main/libs/langchain-cloudflare/src/embeddings.ts.

No functionality here will be lost, so perhaps that improvement would be better served in another PR?

Cherry avatar May 13 '24 22:05 Cherry

As far as I can tell, this is ready to merge. Is there anything left to resolve before merging?

Cherry avatar Aug 28 '24 17:08 Cherry

Apologies will have a look as soon as I can!

jacoblee93 avatar Aug 28 '24 18:08 jacoblee93

Hey team, I have an issue that might be related to this!

image

I'm looking forward to this PR!

wanoo21 avatar Sep 02 '24 11:09 wanoo21

@Cherry is attempting to deploy a commit to the LangChain Team on Vercel.

A member of the Team first needs to authorize it.

vercel[bot] avatar Sep 02 '24 21:09 vercel[bot]

Hey, extremely sorry about the delay, this looks good!

jacoblee93 avatar Sep 10 '24 23:09 jacoblee93

We are about to push minor versions of all packages related to the 0.3 release, will push this out alongside that.

jacoblee93 avatar Sep 10 '24 23:09 jacoblee93