ExpressJS: streamToResponse doen't return a continious text stream

[yescine]

I' am using the ai sdk with a MERN stack app NodeJs v18.16.0 and React v18.2.0, in the express route handler I get the response in one block (with no character stream)

I' am using the provided front-end example (in React set-up) and my NodeJs back-end Code, Any way to debug the text character stream ?

import { Router } from 'express';
import { streamToResponse, OpenAIStream } from 'ai';
import { openai } from './index';
import projectModel from '../../models/project';

const router = Router();

router.post('/project', async (req, res, next) => {
  console.log({ 'ai-project': req.body });
  const { prompt, projectId }:{prompt:string,projectId:string} = req.body;

  const project = await projectModel.findById(projectId).select('tasks');

  const sortedTasks = project.tasks.sort((a, b) => b.createdAt - a.createdAt);
  const lastTasks = sortedTasks.slice(-5);
  const tasksArray = lastTasks.map(task => `notes:(${task.notes})`) as string[];

  const aiResponse = await openai.createChatCompletion({
    model: 'gpt-3.5-turbo',
    messages: [
      {
        role: 'system',
        content:
          'You are an AI project manager assistant ' +
          '["subject:(text) - assignee:(text) - notes:(text)"]' +
          'Limit your response to no more than 150 characters'+

      },
      {
        role: 'user',
        content: `${prompt}\n${JSON.stringify(tasksArray)}`,
      },
    ],
    temperature: 0.8,
    stream: true,
  });

  const stream = OpenAIStream(aiResponse);
  streamToResponse(stream, res);

});

export default router;

[MaxLeiter]

How are you hosting your code? Your provider needs to support streaming responses.

[yescine]

@MaxLeiter the issue is the same locally and on the cloud (docker container in Heroku), I don' t have the issue with a NextJs set-up.

[pangao66]

same question https://github.com/vercel-labs/ai/issues/239

[kstraszewski]

Im having te same problem on Nuxt3, when i deploy it to vercel. When i locally host my app everything works great, after deploying it to Vercel typewriter animation disapear :/

[BEpresent]

Same problem for me, streamToResponse does not stream for me when I host an express app on Vercel - locally I get it working. A working example with express independent of a frontend would be useful

[max-sym]

It's still happening for me as well. Did anyone solve it successfully?

[lgrammel]

Here is a node.js server example: https://github.com/vercel/ai/tree/main/examples/node-http-server

For troubleshooting deployments:

• https://sdk.vercel.ai/docs/troubleshooting/common-issues/streaming-not-working-on-vercel
• https://sdk.vercel.ai/docs/troubleshooting/common-issues/streaming-not-working-on-vercel-pages-router