ai icon indicating copy to clipboard operation
ai copied to clipboard

LangChainStream: add support for function_call streaming

Open davidfant opened this issue 1 year ago • 1 comments

I have a GPT-like chat experience in my product where I want to stream function calls using langchain rather than just regular messages. This PR enables function_call streaming and relies on https://github.com/vercel/ai/blob/main/packages/core/shared/call-chat-api.ts#L113 which picks up function_calls from e.g. OpenAI streaming and constructs the messages correctly on the frontend

I have been using this code in my own codebase, but am not able to build the Vercel AI and thus haven't tested it by building and linking ai. This is what I've tried:

pnpm install
pnpm build

The errors I get are related to crypto not being resolved:

next-google-generative-ai:build: Failed to compile.
next-google-generative-ai:build: 
next-google-generative-ai:build: ../../packages/core/dist/index.mjs:1793:0
next-google-generative-ai:build: Module not found: Can't resolve 'crypto'
next-google-generative-ai:build:   1791 |
next-google-generative-ai:build:   1792 | // ../../node_modules/.pnpm/[email protected]/node_modules/uuid/dist/esm-node/rng.js
next-google-generative-ai:build: > 1793 | import crypto from "crypto";
next-google-generative-ai:build:   1794 | var rnds8Pool = new Uint8Array(256);
next-google-generative-ai:build:   1795 | var poolPtr = rnds8Pool.length;
next-google-generative-ai:build:   1796 | function rng() {
next-google-generative-ai:build: 
next-google-generative-ai:build: https://nextjs.org/docs/messages/module-not-found
next-google-generative-ai:build: 
next-google-generative-ai:build: Import trace for requested module:
next-google-generative-ai:build: ./app/api/completion/route.ts
next-google-generative-ai:build: ../../node_modules/.pnpm/[email protected][email protected][email protected]/node_modules/next/dist/build/webpack/loaders/next-edge-app-route-loader/index.js?absolutePagePath=private-next-app-dir%2Fapi%2Fcompletion%2Froute.ts&page=%2Fapi%2Fcompletion%2Froute&appDirLoader=bmV4dC1hcHAtbG9hZGVyP25hbWU9YXBwJTJGYXBpJTJGY29tcGxldGlvbiUyRnJvdXRlJnBhZ2U9JTJGYXBpJTJGY29tcGxldGlvbiUyRnJvdXRlJnBhZ2VQYXRoPXByaXZhdGUtbmV4dC1hcHAtZGlyJTJGYXBpJTJGY29tcGxldGlvbiUyRnJvdXRlLnRzJmFwcERpcj0lMkZVc2VycyUyRmZhbnQlMkZyZXBvcyUyRmFpJTJGdmVyY2VsLWFpJTJGZXhhbXBsZXMlMkZuZXh0LWdvb2dsZS1nZW5lcmF0aXZlLWFpJTJGYXBwJmFwcFBhdGhzPSUyRmFwaSUyRmNvbXBsZXRpb24lMkZyb3V0ZSZwYWdlRXh0ZW5zaW9ucz10c3gmcGFnZUV4dGVuc2lvbnM9dHMmcGFnZUV4dGVuc2lvbnM9anN4JnBhZ2VFeHRlbnNpb25zPWpzJmJhc2VQYXRoPSZhc3NldFByZWZpeD0mbmV4dENvbmZpZ091dHB1dD0mcHJlZmVycmVkUmVnaW9uPSZtaWRkbGV3YXJlQ29uZmlnPWUzMCUzRCE%3D&nextConfigOutput=&preferredRegion=&middlewareConfig=e30%3D!
next-google-generative-ai:build: 
next-google-generative-ai:build: 
next-google-generative-ai:build: > Build failed because of webpack errors
next-google-generative-ai:build:  ELIFECYCLE  Command failed with exit code 1.
   Generating static pages (1/5) 
next-google-generative-ai:build: ERROR: command finished with error: command (/Users/fant/repos/ai/vercel-ai/examples/next-google-generative-ai) pnpm run build exited (1)
solidstart-openai:build:  ELIFECYCLE  Command failed.
sveltekit-openai:build:  ELIFECYCLE  Command failed.
next-perplexity:build: cache miss, executing 98ea7e0200ffd2bf
nuxt-langchain:build: cache miss, executing 90e533bc8181c341
next-replicate:build: cache miss, executing ecc9c7b3cad45560
nuxt-openai:build: cache miss, executing 070d57fa6a3eeb26
next-mistral:build: cache miss, executing d0293dfecc9759ee
next-anthropic:build: cache miss, executing b3fe53e96c161577
next-aws-bedrock:build: cache miss, executing 06dddc31afd8dcf7
next-google-generative-ai#build: command (/Users/fant/repos/ai/vercel-ai/examples/next-google-generative-ai) pnpm run build exited (1)

 Tasks:    1 successful, 18 total
Cached:    1 cached, 18 total
  Time:    8.72s 
Failed:    next-google-generative-ai#build

 ERROR  run failed: command  exited (1)
 ELIFECYCLE  Command failed with exit code 1.

davidfant avatar Jan 27 '24 11:01 davidfant

Any maintainer that can take a look at this?

davidfant avatar Feb 13 '24 12:02 davidfant