[Bug]: Streaming issue with parallel results, multiple providers - response flashes blank and then back

[cloud-25683]

What happened?

when enabling multiConvo (multiConvo: true) and compare two different AI provider models, the right hand side response is not being rendered on the UI (firefox and chrome), however, response did come back, if I click the copy button under the response and paste in a notepad, it is there (sometimes it even shows up for a split of second and then disappeared from UI), so the issue is UI rendering, and only happens when comparing Azure OpenAI vs Bedrock or Bedrock vs Azure OpenAI, the right side one is not being rendered, but it works for the same provider comparison, for example, compare gpt-4o vs gpt-4o-mini (I have attached 4 screenshots here to illustrate the issue)

Steps to Reproduce

select any mode from bedrock, click multi-convo button and then select an azure openai model, run it, the right side response is not being rendered on the UI

What browsers are you seeing the problem on?

No response

Relevant log output

No response

Screenshots

Code of Conduct

• [X] I agree to follow this project's Code of Conduct

[cloud-25683]

@danny-avila curious if you have seen this as well? it's a very strange UI rendering issue, let me know if u have any insights, thank you

[illgitthat]

@cloud-25683 can you please change the title to indicate that this is not specific to the AI providers you mentioned.

Also, sharing video recordings from #6019 in case it helps.

https://github.com/user-attachments/assets/8485a0f7-abc0-4129-96ce-0d03b9f99533

https://github.com/user-attachments/assets/1a39d900-646c-4fac-9ae4-c7a9e23ccb23

@danny-avila if you are able to provide any guidance on what areas of the code might be affecting this so I could take a look at a fix, please let me know.

[rtamang21]

any update on this issue?

[dannykorpan]

Same issue here, but the complete answer doesn't come back up.

[K0IN]

I have this issue nearly every time i use multiple llms (claude and gpt 4.1) on latest version

[maholick]

I have the same issue. In this example Mistral starts with output and then stops / clearing the output in UI. On refresh the output appears. The issue happens with different model combinations but with Claude I think its "worse".

When stopping the generation with Claude & Mistral for instance, the output comes back but Claude stopped in between.

https://github.com/user-attachments/assets/6d5947cf-072a-4caf-98a4-ffdebf5ca589

[rfe-sdev]

Here's a quick fix so that the answers are always displayed at the end. It doesn't fix the streaming blink problem, but at least the answers are there!

Replace the onStart method in the abortMiddleware.js file by the following :

const onStart = (userMessage, responseMessageId) => {
    sendMessage(res, { message: userMessage, created: true });

    const abortKey = `${userMessage?.conversationId ?? req.user.id}:${responseMessageId}`;
    getReqData({ abortKey });

    // Store minimal options
    const minimalOptions = {
      endpoint: endpointOption.endpoint,
      iconURL: endpointOption.iconURL,
      model: endpointOption.modelOptions?.model || endpointOption.model_parameters?.model,
    };

    abortControllers.set(abortKey, { abortController, ...minimalOptions });

    // Use a simple function for cleanup to avoid capturing context
    const cleanupHandler = () => {
      try {
        cleanupAbortController(abortKey);
      } catch (e) {
        // Ignore cleanup errors
      }
    };

    res.on('finish', cleanupHandler);
  };

I suspect that when two requests are launched simultaneously for the same conversation, the second request sometimes overwrites the first's abort controller in the abortControllers Map. I have therefore improved the uniqueness of the keys by using the responseMessageId.

⚠️ I warn you that I'm not sure of all the implications of this.

[kmilo9999]

Hello, I am using librechat v.0.8.0 and this problem still happens. The code of abortMiddleware.js was modified recently in their last release, so I am not sure if @rfe-sdev suggestion still applies. Are there any guidance to fix this issue?

[cazzerson]

Also seeing this in latest dev containers as of today.