generate assistant message id from backend side

[himself65]

Feature Description

We have a chat backend with persistence layer, so everytime user refresh page will keep track of the history messages. So it's necessary that keep message id constant everytime user popup a new message.

But I found that in StreamResponse protocol there is only possible to return text and data. But there's no id. and the reason of generateId is not working because it means id is generated from frontend, but for the ai message, it's actually generated from backend. So it won't work.

Use Case

I have some features that will allow user to custimize ai chat message per different browser session. Like double click edit, mark, quote, and index sources. So it's important to keep id constant with backend.

Additional context

my current workaround

  onFinish: useCallback(
      (message: Message) => {
        // handle the AI message generated by the backend, because id is not returned in the response,
        //  we need to fetch the latest session to get the id of the last message
        getChatSessionFromBackend({
          sessionId,
        })
          .then((newSession) => {
            const lastAIMessage = newSession.chat_messages
              ?.sort((a, b) => (a.index > b.index ? 1 : -1))
              .at(-1);
            if (lastAIMessage?.role !== "assistant") {
              throw unexpected()
              return;
            } else {
              const id = lastAIMessage.id;
              storeMessage({ ...message, id }) // store message to indexeddb
              });
            }
          })
          .then(async () => {
            await refetch();
          })
      },
      [pipelineId, refetch],
    ),

[himself65]

I think the streaming protocol should allow gives the id:id for the first chunk

[LucasYver]

Hey,

Can't you use the 2: custom JSON added by the user using Data using toDataStream ?

    const data = new StreamData()
    data.append('my-id')
    const reader = this.stream_result.toDataStream({ data }).getReader()

Like this the first chunk will be your custom json.

[etburke]

I'd also find this feature useful!

[himself65]

Thanks for doing this, I will try it