openai-like api endpoint doesn't work well as before in b1382

[gaord]

Prerequisites

Please answer the following questions for yourself before submitting an issue.

• [x] I am running the latest code. Development is very rapid so there are no tagged versions as of now.
• [x] I carefully followed the README.md.
• [x] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).
• [x] I reviewed the Discussions, and have a new bug or useful enhancement to share.

Expected Behavior

I can setup an openai-like api endpoint with server and api_like_OAI.py as in readme instruction

Current Behavior

chating through openai-like api endpoint with streaming results in "sse error: Request failed with status code 500". Old version as of b1382 can work just fine.

here is web app output while calling the endpoint: stack: 'Error: Request failed with status code 500\n' + ' at createError (E:\aigc\gitFastGpt\client\node_modules\openai\node_modules\axios\lib\core\createError.js:16:15)\n' + ' at settle (E:\aigc\gitFastGpt\client\node_modules\openai\node_modules\axios\lib\core\settle.js:17:12)\n' + ' at RedirectableRequest.handleResponse (E:\aigc\gitFastGpt\client\node_modules\openai\node_modules\axios\lib\adapters\http.js:278:9)\n' + ' at RedirectableRequest.emit (node:events:525:35)\n' + ' at RedirectableRequest._processResponse (E:\aigc\gitFastGpt\client\node_modules\follow-redirects\index.js:368:10)\n' + ' at RedirectableRequest._onNativeResponse (E:\aigc\gitFastGpt\client\node_modules\follow-redirects\index.js:65:10)\n' + ' at Object.onceWrapper (node:events:628:26)\n' + ' at ClientRequest.emit (node:events:513:28)\n' + ' at HTTPParser.parserOnIncomingClient (node:_http_client:701:27)\n' + ' at HTTPParser.parserOnHeadersComplete (node:_http_common:119:17)\n' + ' at Socket.socketOnData (node:_http_client:542:22)\n' + ' at Socket.emit (node:events:513:28)\n' + ' at addChunk (node:internal/streams/readable:324:12)\n' + ' at readableAddChunk (node:internal/streams/readable:297:9)\n' + ' at Readable.push (node:internal/streams/readable:234:10)\n' + ' at TCP.onStreamRead (node:internal/stream_base_commons:190:23)',

[wendlerc]

also does not work for me

[GWFJ]

same, it doesn't work off the latest server-cuda docker image.

[github-actions[bot]]

This issue was closed because it has been inactive for 14 days since being marked as stale.