Assistants Multiple Response Loading Bug

[jackitaliano]

Scenario

• deep-chat v1.4.11, also appears to be present in deep-chat-dev v9.0.156 (so not likely version dependent)
• directConnection to OpenAI Assistants
• load_thread_history enabled, code_interpreter enabled
• When assistant returns multiple responses, only 1 message is added.

Recreate

there are other ways to recreate, this one seems most consistent

• Ask assistant to give example data for a csv and create a suitable bar chart for it (in one message)
• One text response is given with the corresponding bar chart image. Upon refresh (with load thread history) a second text response is usually present.

can also make this happen by uploading your own csv and then asking for the same

Thoughts

It seems that the run is complete when the message is returned, so I'm guessing it's not an issue of not waiting for the run to finish.

In that case, my guess is that multiple text responses may not be handled. I.e. deep-chat has the message, but only adds the most recent message from the thread instead of all the recents from the latest run. Could very well be wrong.

Not incredibly urgent, but hoping for a fix.

Thanks!

[OvidijusParsiunas]

Deep Chat only looks at the last message from that thread. The fix for this should be quite simple, but I will exercise caution. This also does not appear to work for streams, though the solution for that might be much more complex. I will update you on the progress. Thanks!

[jackitaliano]

Deep Chat only looks at the last message from that thread. The fix for this should be quite simple, but I will exercise caution. This also does not appear to work for streams, though the solution for that might be much more complex. I will update you on the progress. Thanks!

That's kinda what I suspected. Again, no rush, especially with the new job you mentioned in other issue.

Thanks!

[OvidijusParsiunas]

Hi, I have managed to fix this for regular requests in deep-chat-dev and deep-chat-react-dev version 9.0.159. Please let me know if you see any issues. I will investigate the solution for streaming tomorrow. Thanks!

[OvidijusParsiunas]

Hi, I have now updated the deep-chat-dev and deep-chat-react-dev packages to version 9.0.160 to allow streaming with files. Let me know if this works for you. Thanks!

[jackitaliano]

After testing this out, there seems to be an issue that this fix creates.

When the assistant returns an image file, it returns multiple of the same image. I've tried recreating this on prod, and this issue does not appear.

Recreate

1. Send message "Give example data for a csv and write it to a file. Then make a suitable bar chart for the example data."
2. Assistant returns response with text, one csv, and two of the same images.

• can put a console.log(response) in response interceptor, which shows that the response has a duplicate image.
• this can also be seen with a console.log(message) in onMessage or onNewMessage (depending on version)

This appears in deep-chat-dev v9.0.160, but does not appear in deep-chat v1.4.11.

[OvidijusParsiunas]

Hey @jackitaliano.

I can see the issue. I'm a little out of time today but I'll investigate it as soon as I get a chance tomorrow. Thanks!

[jackitaliano]

Hello @OvidijusParsiunas ,

Sounds good, thanks for looking into!

[OvidijusParsiunas]

Hi @jackitaliano.

The issue should be fixed in version 9.0.165. Let me know if this works for you and if you encounter any issues. Thanks!

[jackitaliano]

Hello @OvidijusParsiunas,

This issue appears to still be present.

In this chat, there are only the two files (image and csv), but it loads the image twice.

Hope this isn't too much of a hastle.

Thanks!

Edit: To be clearer, this issue doesn't seem to be an edge case. If you make the same request as the user, it'll usually respond like that. Also, the same occurs if there are multiple text responses.

[OvidijusParsiunas]

Hi @jackitaliano.

Thankyou for letting me know about this issue. Deep Chat has been configured to parse the incoming response message and if it identifies a reference to a file it automatically creates another message with that file to allow the user to download. However it appears that OpenAI assistants are automatically sending another message with the file contents for images. I was able to fix this problem and have released it in version 9.0.167. I have also noticed that when there are multiple response messages, the messages (except the last one) are displayed in the wrong order. This has also been fixed.

Through testing I have also found that when streaming, the response can sometime stop and not generate other messages e.g. the bar chart message. It appears this happens when there is a "server error" response. Hence, unfortunately this is on the OpenAI side of things, so if the stream does not return the full result - Deep Chat can't do much about it.

Thankyou for your feedback and let me know if you notice any other issues in the future.

[OvidijusParsiunas]

Hi, the functionality mentioned in this issue has now been released in Deep Chat version 2.0.0.