genkit icon indicating copy to clipboard operation
genkit copied to clipboard

[JS] Gemini JSON mode can't stream

Open mbleigh opened this issue 1 year ago • 1 comments

Describe the bug When generating in JSON mode, the Gemini plugin crashes when trying to send partial responses. This is because we use standard JSON.parse instead of a lenient parser.

To Reproduce

const {stream} = generateStream({
  prompt: "Generate a list of 50 contacts for synthetic data testing.",
  output: {format: "json", schema: z.object({people: z.array(z.object({name: z.string(), email: z.string()}))})},
});

for await (const chunk of stream()) {
  // crashes here
  console.log(chunk);
}

Expected behavior A partial object is returned as a chunk.

mbleigh avatar Sep 25 '24 03:09 mbleigh

Still needs fixing in next

cabljac avatar Oct 03 '24 15:10 cabljac