bug: Error thrown when > 4 stop words sent

[sestinj]

Describe the bug When the "stop" parameter is used with more than 4 strings, Jan returns an error. Logs are attached below

Steps to reproduce Steps to reproduce the behavior:

1. Try the following curl request with 4 stop words, it will succeed:

curl http://localhost:1337/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $OPENAI_API_KEY" \
     -d '{
   "model": "gpt-3.5-turbo",
   "messages": [
     {
       "role": "system",
       "content": "You are a helpful assistant."
     },
     {
       "role": "user",
       "content": "Hello!"
     }
   ], "stop": ["<end>", "<END>", "<|end|>", "<|END|>"]
 }'

2. Then add one more stop word to make it 5. This will throw an error:

```bash
curl http://localhost:1337/v1/chat/completions \
     -H "Content-Type: application/json" \
     -H "Authorization: Bearer $OPENAI_API_KEY" \
     -d '{
   "model": "gpt-3.5-turbo",
   "messages": [
     {
       "role": "system",
       "content": "You are a helpful assistant."
     },
     {
       "role": "user",
       "content": "Hello!"
     }
   ], "stop": ["<end>", "<END>", "<|end|>", "<|END|>", "END"]
 }'

Expected behavior OpenAI's reference specifies that they only support 4 stop words, but they automatically cut off any extras. Ideally, Jan would do the same. It is problematic for us to do it client side because there are implementations of the OpenAI API that actually accept more than 4 stop words and we want users to benefit from this without needing to write in a config file somewhere that the API they use supports this.

Environment details

• Operating System: Apple M1 Pro, Sonoma 14.3
• Jan Version: 0.4.7

Logs

{"level":30,"time":1709327400898,"pid":61362,"hostname":"dhcp-10-29-238-127.dyn.MIT.EDU","reqId":"req-6","req":{"method":"POST","url":"/v1/chat/completions","hostname":"localhost:1337","remoteAddress":"127.0.0.1","remotePort":58507},"msg":"incoming request"}
{"level":30,"time":1709327401559,"pid":61362,"hostname":"dhcp-10-29-238-127.dyn.MIT.EDU","reqId":"req-6","res":{"statusCode":200},"responseTime":660.7822079658508,"msg":"request completed"}
{"level":30,"time":1709327499636,"pid":61362,"hostname":"dhcp-10-29-238-127.dyn.MIT.EDU","reqId":"req-7","req":{"method":"POST","url":"/v1/chat/completions","hostname":"127.0.0.1:1337","remoteAddress":"127.0.0.1","remotePort":58553},"msg":"incoming request"}
{"level":30,"time":1709327499852,"pid":61362,"hostname":"dhcp-10-29-238-127.dyn.MIT.EDU","reqId":"req-7","res":{"statusCode":400},"err":{"type":"TypeError","message":"The \"readableStream\" argument must be an instance of ReadableStream. Received an instance of PassThrough","stack":"TypeError [ERR_INVALID_ARG_TYPE]: The \"readableStream\" argument must be an instance of ReadableStream. Received an instance of PassThrough\n    at new NodeError (node:internal/errors:405:5)\n    at Object.newStreamReadableFromReadableStream (node:internal/webstreams/adapters:482:11)\n    at Readable.fromWeb (node:internal/streams/readable:1412:27)\n    at sendWebStream (/Applications/Jan.app/Contents/Resources/app.asar/node_modules/fastify/lib/reply.js:706:31)\n    at onSendEnd (/Applications/Jan.app/Contents/Resources/app.asar/node_modules/fastify/lib/reply.js:668:7)\n    at onSendHook (/Applications/Jan.app/Contents/Resources/app.asar/node_modules/fastify/lib/reply.js:557:5)\n    at Reply.send (/Applications/Jan.app/Contents/Resources/app.asar/node_modules/fastify/lib/reply.js:178:7)\n    at /Applications/Jan.app/Contents/Resources/app.asar/node_modules/@janhq/core/dist/node/index.cjs.js:1663:37\n    at step (/Applications/Jan.app/Contents/Resources/app.asar/node_modules/@janhq/core/dist/node/index.cjs.js:110:23)\n    at Object.next (/Applications/Jan.app/Contents/Resources/app.asar/node_modules/@janhq/core/dist/node/index.cjs.js:91:53)","code":"ERR_INVALID_ARG_TYPE"},"msg":"The \"readableStream\" argument must be an instance of ReadableStream. Received an instance of PassThrough"}
{"level":30,"time":1709327499853,"pid":61362,"hostname":"dhcp-10-29-238-127.dyn.MIT.EDU","reqId":"req-7","res":{"statusCode":400},"responseTime":217.27470803260803,"msg":"request completed"}

[louis-jan]

Thank you for the great find, @sestinj. We will check and cut out the extra stop words accordingly.

[louis-jan]

Hi @sestinj, it's to let you know that we have fixed the issue in the 0.4.10 release. Please let us know if it does not work as expected.

[sestinj]

Works perfectly, thank you!!