CodeGPT icon indicating copy to clipboard operation
CodeGPT copied to clipboard

Custom OpenAI autocomplete causes IDE error when "stream" is set to false

Open evrenesat opened this issue 1 year ago • 2 comments

What happened?

When its set to connect LM Studio and when the stream is "true" it works, when it's false, it fails with "Unknown API response" error.

Relevant log output or stack trace

Unknown API response. Code: 200, Body: {
  "id": "cmpl-dq18edmm0uqpmsf2xuia",
  "object": "text_completion",
  "created": 1736928943,
  "model": "qwen2.5-coder-7b-instruct-mlx",
  "choices": [
    {
      "index": 0,
      "text": "\n\n*/\n\n convert it.\n\n    :param value: The value to ensure as a string.\n    :param allow_empty: Whether we should throw an error if `value` is empty.\n    :return: The value as a string.\n    :raises ValueError: If not allow_empty and the `value` is empty.\n    \"\"\"\n\n    if value is None or value == \"\" or value == [] or value == {}:\n        if not allow_empty:\n            raise ValidationError(\"A valid String is required.\")\n        return \"\"\n\n    if isinstance(value, list):\n        results = [ensure_string(item) for item in value if item]\n        return",
      "logprobs": null,
      "finish_reason": "length"
    }
  ],
  "usage": {
    "prompt_tokens": 302,
    "completion_tokens": 127,
    "total_tokens": 429
  }
}

java.lang.RuntimeException
	at ee.carlrobert.llm.completion.CompletionEventSourceListener.onFailure(CompletionEventSourceListener.java:116)
	at okhttp3.internal.sse.RealEventSource.processResponse(RealEventSource.kt:59)
	at okhttp3.internal.sse.RealEventSource.onResponse(RealEventSource.kt:46)
	at okhttp3.internal.connection.RealCall$AsyncCall.run(RealCall.kt:519)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)


### Steps to reproduce

_No response_

### CodeGPT version

2.15.2-241.1

### Operating System

macOS

evrenesat avatar Jan 15 '25 08:01 evrenesat

I have same problem. @carlrobertoh can you help us? For inference use vLLM.

unixshaman avatar Feb 24 '25 12:02 unixshaman

Unfortunately, this error is still present.

unixshaman avatar Sep 03 '25 09:09 unixshaman