Improve SSE User Experience

[VGEAREN]

Why are these changes needed?

[Feature Enhancement] Improve SSE User Experience

This PR aims to enhance the user experience of SSE (Server-Sent Events). The following changes have been made:

• Replace fastapi.StreamingResponse with sse_starlette.sse.EventSourceResponse

Related issue number (if applicable)

Checks

• [x] I've run format.sh to lint the changes in this PR.
• [x] I've included any doc changes needed.
• [x] I've made sure the relevant tests are passing (if applicable).

[alanxmay]

@VGEAREN I have made a similar modification before, but it has a problem that it is not compatible with the openai python sdk, because it will send a ping event every once in a while, which requires special handling by the client. Have you checked out this recently added feature? See OpenAI-Compatible RESTful APIs.

[VGEAREN]

@VGEAREN I have made a similar modification before, but it has a problem that it is not compatible with the openai python sdk, because it will send a ping event every once in a while, which requires special handling by the client. Have you checked out this recently added feature? See OpenAI-Compatible RESTful APIs.

yes, I think use a high ping interval can fix it.

[merrymercy]

@andy-yang-1 Could you test whether this is compatible with openai python package?

[merrymercy]

@VGEAREN I am not very familiar with SSE. Could you educate me on how this is better than StreamingResponse ? Also, does your current PR fully solve the issues mentioned by @alanxmay ?

[VGEAREN]

@VGEAREN I am not very familiar with SSE. Could you educate me on how this is better than StreamingResponse ? Also, does your current PR fully solve the issues mentioned by @alanxmay ?

I integrated the OpenAI API and Fastchat API in a project(JAVA). After completing the OpenAI integration successfully, I attempted to use the same request method for Fastchat. However, I realized that it didn't work as expected. Eventually, I discovered that the issue was with the StreamingResponse. Therefore, I switched to EventSourceResponse, and it worked fine.

And yes, I have fixed the issue.

[merrymercy]

@VGEAREN Thanks. It is merged.

[merrymercy]

Sorry, I have to revert this PR because it introduces some errors listed in #1875. Feel free to fix them and open a new PR if you still want this feature in the future version of FastChat.