Error parsing JSON: unexpected ','; expected end of input

[LyndonArmitage]

I am consistently getting the following error when using the application (even when using the latest Development build):

Error when calling API: Error parsing JSON: [json.exception.parse_error.101] parse error at line 1, column 8: syntax error while parsing value - unexpected ','; expected end of input

Essentially, this happens after the output to a query. Could it be that the OpenAI API is returning non-standard JSON with trailing , symbols? Or that the API output looks different to the expected output?

I think the issue is happening somewhere in handle_streamed_response inside main/interface/network/Network.cpp

Unfortunately, I am unable to compile on my Linux (Manjaro) machine to confirm the cause or find a fix. But perhaps an additional line to remove a trailing , here might help?

//Remove leading and trailing whitespaces and new lines.
s.erase(s.begin(), std::find_if(s.begin(), s.end(), [](unsigned char c){return !std::isspace(c) && c != '\n';}));
s.erase(std::find_if(s.rbegin(), s.rend(), [](unsigned char c){return !std::isspace(c) && c != '\n';}).base(), s.end());

For completeness, my compile error is:

CMake Error at CMakeLists.txt:18 (find_package):
  Could not find a package configuration file provided by "CURL" with any of
  the following names:

    CURLConfig.cmake
    curl-config.cmake

  Add the installation prefix of "CURL" to CMAKE_PREFIX_PATH or set
  "CURL_DIR" to a directory containing one of the above files.  If "CURL"
  provides a separate development package or SDK, be sure it has been
  installed.

I am not super familiar with CMake but I do have cUrl installed on my machine. It might be useful to have a section in the README about what is needed to compile, as this is making me think I need to pass a specific parameter to CMake or provide it with cUrl source code in someway,

[LyndonArmitage]

Regarding building, I was able to move on slightly by removing the CONFIG option from the CMake line for CURL. I had to then install ftxui and am now being hit with CMake errors to do with unofficial-utf8proc. I have the official utf8proc library installed on my system and altering that CMake line to find_package(utf8proc REQUIRED) still gives me an error about cmake package files.

This is all probably a compiling issue on my side, and this issue is primarily related to the Error parsing JSON issue so I won't give any further updates on here about my limited CMake knowledge, but the JSON issue still stands.

[LagPixelLOL]

Lemme test... 🤔

[LagPixelLOL]

Is this happening to you in every request(literally every request) or just some of the requests? Which model are you using? Can you post your config file's content in here(without the API key)? The weird part is it's working for me...

[ElfTheHunter]

Hey, I'm having the same issue. For me it's not every request, but like 95% of them or more. I'm using "gpt-4". Here's my config:

"api_base_url": "https://api.openai.com", "api_key": "******", "debug_reference": false, "frequency_penalty": 0.0, "logit_bias": {}, "max_display_length": 100, "max_reference_length": 4, "max_short_memory_length": 4, "max_tokens": 10000, "model": "gpt-4", "presence_penalty": 0.6000000238418579, "search_response": true, "space_between_exchanges": false, "temperature": 0.8, "top_p": 1.0

It just started yesterday or earlier today.

[ElfTheHunter]

Here's an example, it gives almost the entire response (I asked it to tell me a 12 paragraph story about a viking) then the error comes at the end.

[LagPixelLOL]

Huh... I still can't replicate this...

[ElfTheHunter]

I don't know if this is helpful - this is all way above my technical knowledge but... https://community.openai.com/t/bug-streaming-packets-changed/460882

[ElfTheHunter]

They referenced your CLI at some point:

sevdmr I am experiencing this issue also, using this application: https://github.com/LagPixelLOL/ChatGPTCLIBot/

[atty-openai] It looks like this library is not parsing SSE outputs correctly. It assumes that when the stream content is split by \n\ndata:, each line is valid JSON. However, SSE clients should buffer output until they hit a separator character and only then parse the chunk into JSON.

[LagPixelLOL]

Ok I just replicated it, working on it now...

[LagPixelLOL]

I just pushed an update CLICK HERE, it's not a fix but it now gives more detailed error messages about JSON, can you guys try it out can send the error message here?

[ElfTheHunter]

I copy/pasted the config file from the old folder. First attempt did not give an error, but I tried asking it for another long story outline and it give this:

Error when calling API: Error parsing JSON: [json.exception.parse_error.101] parse error at line 1, column 4: syntax error while parsing value - invalid literal; last read: '","c'; expected end of input Raw response: "","content":""},"finish_reason":null}]}" Processed response: "","content":""},"finish_reason":null}]}"

[LyndonArmitage]

Thanks for the speedy response @LagPixelLOL

Very weird that you could not initially reproduce it. Perhaps there's some A/B testing going on with the API. I am based in the UK and am using GPT-3.5.

The following is the error I just got:

Error when calling API: Error parsing JSON: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'r'
Raw response: "role":"assistant","content":""},"finish_reason":null}]}"
Processed response: "role":"assistant","content":""},"finish_reason":null}]}"

EDIT: I was able to get another, slightly different error by prompting the AI with "Say 10":

Error when calling API: Error parsing JSON: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'o'
Raw response: "ole":"assistant","content":""},"finish_reason":null}]}"
Processed response: "ole":"assistant","content":""},"finish_reason":null}]}"

This is looking like some kind of issue with the way chunks of the response are being processed, given that these 3 errors (including the one from @ElfTheHunter) seems slightly different.

[LyndonArmitage]

Here is my config file:

{
  "api_base_url": "https://api.openai.com",
  "api_key": "sk-***",
  "ca_bundle_path": "/etc/ssl/certs/ISRG_Root_X2.pem",
  "debug_reference": false,
  "frequency_penalty": 0.0,
  "logit_bias": {},
  "max_display_length": 100,
  "max_reference_length": 4,
  "max_short_memory_length": 4,
  "max_tokens": 10000,
  "model": "gpt-3.5-turbo",
  "presence_penalty": 0.6000000238418579,
  "search_response": true,
  "space_between_exchanges": false,
  "temperature": 0.8,
  "top_p": 1.0
}

[ElfTheHunter]

I decided to try a default config, to see if it still gives the same error. Here's the result:

"api_base_url": "https://api.openai.com", "api_key": "*****", "debug_reference": false, "frequency_penalty": 0.0, "logit_bias": {}, "max_display_length": 100, "max_reference_length": 4, "max_short_memory_length": 4, "max_tokens": 500, "model": "gpt-3.5-turbo", "presence_penalty": 0.0, "search_response": true, "space_between_exchanges": false, "temperature": 1.0, "top_p": 1.0

Error when calling API: Error parsing JSON: [json.exception.parse_error.101] parse error at line 1, column 1: syntax error while parsing value - invalid literal; last read: 'o' Raw response: "ole":"assistant","content":""},"finish_reason":null}]}" Processed response: "ole":"assistant","content":""},"finish_reason":null}]}"

[LagPixelLOL]

It's weird that I can hardly reproduce it, but I'll try more!

[ElfTheHunter]

I don't know if this helps, but it seems like the first word of every response is missing. Instead of responding with "Once upon a time..." it instead starts " upon a time..." then it errors out before completing the response.

[LyndonArmitage]

As I said in an earlier comment, I think it's to do with how the streamed response is being parsed. In handle_streamed_response there's a split happening using a regex.

        std::vector<std::string> split_str;
        boost::split_regex(split_str, std::string(raw_vec.begin(), raw_vec.end()), boost::regex("\\n\\ndata: *"));
        for (auto& raw_str : split_str) {

If, in the throw statement, you output the bad entry in the vector along with all other raw entries in split_str it might give a clue as to what is happening. My theory is that the JSON response has been split up amongst the chunks for some reason. This could be to do with the regex, but solving it might be difficult.

I really wish I could help more, but I can't get this to compile on my machine with ease :neutral_face: and my C++ is very rusty. I will try and make a Dockerfile to build it when I have a spare hour, then I can debug right on my machine with more ease.

[LagPixelLOL]

@LyndonArmitage I know what's happening, it's not with the regex but OpenAI giving SSE in small chunks, I will fix this some day but I'm very busy at the moment...

[LyndonArmitage]

Awesome, thanks for the time you've spent on it so far :smile: I look forward to the fix when you have a chance as this is a brilliant application.

[ElfTheHunter]

I know you're busy and will work on this when you have time, but in the meantime, is there any suggestions on what I can do on my end to minimize or avoid the error? any config settings/prompts/etc - I've found nothing out there that matches how great this CLI is (particularly in regards to long term memory). Kudos on the fantastic work btw!

[LagPixelLOL]

I know you're busy and will work on this when you have time, but in the meantime, is there any suggestions on what I can do on my end to minimize or avoid the error? any config settings/prompts/etc - I've found nothing out there that matches how great this CLI is (particularly in regards to long term memory). Kudos on the fantastic work btw!

Hello, you can try either using SillyTavern or using a reverse proxy. SillyTvern is a UI for mainly roleplaying but it does have long term memory. A reverse proxy is just an API on top of OpenAI's official API that redirects the requests to it and MAY not have the issue with the official API, but you need to be careful because some of them might steal your key, or you may need to host it by yourself. Either way, I'll link you to both.

SillyTavern: https://github.com/SillyTavern/SillyTavern Reverse API(You need to host it by yourself): https://github.com/songquanpeng/one-api

[ElfTheHunter]

Much appreciated, I'll give them a try.

[LagPixelLOL]

NOTICE: NO LONGER MAINTAINED