bug: internal server error [500] Converting circular structure to JSON

[mog13]

Describe the bug When using the local API server and making a call to 'chat/completions' using any Claude or Mistral I get an instant 500 error complaining about circular JSON. I have tried with combinations of different parameters (including trying both streaming and non streaming)

This doesn't occur with any of the OpenAI models. This doesn't occur when using threads inside Jan itself.

This may be related to #2895 but there is no timedelay and doesn't seem to be affected by streaming being on or off. (that ticket also references #1852 which has been fixed and closed)

Steps to reproduce Steps to reproduce the behavior:

1. Start local API server 2. Make a POST to /chat/completions using something like mistral-large-latest or claude-3-opus-20240229

curl --request POST \
  --url http://localhost:7654/v1/chat/completions \
  --header 'Content-Type: application/json' \
  --header 'User-Agent: insomnia/2023.5.8' \
  --data '{
  "messages": [
    {
      "content": "Hello! please write me a haiku",
      "role": "user"
    }
  ],
  "model": "mistral-large-latest",
  "max_tokens": 2048,
  "stop": ["<|end_of_text|>","<|eot_id|>"],
	"stream": true,
  "frequency_penalty": 0,
  "presence_penalty": 0,
  "temperature": 0.7,
  "top_p": 0.95
}'

(you can omit all but model and message and see the same behaviour)

3. Observe 500 response

Expected behavior a non 500 response

Screenshots

Environment details

• Operating System: Windows 11 Pro 64-bit
• Jan Version: 0.4.13 (also tried 0.4.13-451)
• Processor: AMD Ryzen 9 3900X
• RAM: 32GB
• Graphics: NVIDIA GeForce GTX 1070

Logs

[SERVER]::{"reqId":"req-8","res":{},"req":{"method":"POST","url":"/v1/chat/completions","hostname":"localhost:7654"},"msg":"incoming request"}
2024-05-24T20:48:18.641Z [SERVER]::{"reqId":"req-8","res":{"statusCode":500},"req":{},"msg":"request completed","responseTime":9.3082000000868}