genkit icon indicating copy to clipboard operation
genkit copied to clipboard

[JS] Issues with handling streamed output from `generateStream`

Open chrisraygill opened this issue 1 year ago • 0 comments

The docs say that the way to handle streamed output from generateStream is as follows:

import { generateStream } from "@genkit-ai/ai";
import { GenerateResponseChunk } from "@genkit-ai/ai/lib/generate";

const llmResponseStream = await generateStream({
  prompt: 'Suggest a complete menu for a pirate themed restaurant',
  model: gemini15Flash,
});

for await (const responseChunkData of llmResponseStream.stream()) {
  const responseChunk = responseChunkData as GenerateResponseChunk;
  console.log(responseChunk.text());
}

However, import { GenerateResponseChunk } from "@genkit-ai/ai/lib/generate" is a bad import. Additionally, this makes the syntax for handling the streaming unnecessarily complicated. Logging the streamed output to console should be as simple as:

for await (const responseChunkData of llmResponseStream.stream()) {
  console.log(responseChunkData.text());
}

chrisraygill avatar Sep 17 '24 21:09 chrisraygill