jxnl
Intermittent Invalid JSON error GPT4-Vision
- [ ] This is an actually a bug report.
- [ ] I am not getting good LLM Results
- [ ] I have tried asking for help in the community on discord or discussions and have not received a response.
- [x ] I have tried searching the documentation and have not found an answer.
What Model are you using?
- [ ] gpt-3.5-turbo
- [ ] gpt-4-turbo
- [ ] gpt-4
- [ gpt-4-vision-preview] Other (please specify)
Describe the bug The test below fails intermittently, while extracting the model
To Reproduce Runing a
Expected behavior Test succeeds successfully in a consistent fashion.
=================================== FAILURES ===================================
___________________ TestVLLM.test_analyze_image_with_anomaly ___________________
self = <test_vllm.TestVLLM object at 0x7f91a2ad7c10>
tests/test_vllm.py:90:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
vllm/gpt4_vllm.py:50: in analyze_image
report = self.client.chat.completions.create(
.venv/lib/python3.10/site-packages/instructor/patch.py:588: in new_create_sync
response = retry_sync(
.venv/lib/python3.10/site-packages/instructor/patch.py:405: in retry_sync
for attempt in max_retries:
.venv/lib/python3.10/site-packages/tenacity/__init__.py:347: in __iter__
do = self.iter(retry_state=retry_state)
.venv/lib/python3.10/site-packages/tenacity/__init__.py:325: in iter
raise retry_exc.reraise()
.venv/lib/python3.10/site-packages/tenacity/__init__.py:158: in reraise
raise self.last_attempt.result()
../../../.pyenv/versions/3.10.12/lib/python3.10/concurrent/futures/_base.py:451: in result
return self.__get_result()
../../../.pyenv/versions/3.10.12/lib/python3.10/concurrent/futures/_base.py:403: in __get_result
raise self._exception
.venv/lib/python3.10/site-packages/instructor/patch.py:459: in retry_sync
raise e
.venv/lib/python3.10/site-packages/instructor/patch.py:420: in retry_sync
return process_response(
.venv/lib/python3.10/site-packages/instructor/patch.py:213: in process_response
model = response_model.from_response(
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
cls = <class 'common.shared_resources.ReportDetails'>
completion = ChatCompletion(id='chatcmpl-8xK6SmmxPcvnthvu75aIgI9bjn59d', choices=[Choice(finish_reason='stop', index=0, logprobs=No...ompletion', system_fingerprint=None, usage=CompletionUsage(completion_tokens=0, prompt_tokens=1331, total_tokens=1331))
validation_context = None, strict = None
mode = <Mode.MD_JSON: 'markdown_json_mode'>
@classmethod
def from_response(
cls,
completion: T,
validation_context: Optional[Dict[str, Any]] = None,
strict: Optional[bool] = None,
mode: Mode = Mode.TOOLS,
) -> Dict[str, Any]:
"""Execute the function from the response of an openai chat completion
Parameters:
completion (openai.ChatCompletion): The response from an openai chat completion
throw_error (bool): Whether to throw an error if the function call is not detected
validation_context (dict): The validation context to use for validating the response
strict (bool): Whether to use strict json parsing
mode (Mode): The openai completion mode
Returns:
cls (OpenAISchema): An instance of the class
"""
assert hasattr(completion, "choices")
if completion.choices[0].finish_reason == "length":
raise IncompleteOutputException()
message = completion.choices[0].message
if mode == Mode.FUNCTIONS:
assert (
message.function_call.name == cls.openai_schema["name"] # type: ignore[index]
), "Function name does not match"
return cls.model_validate_json(
message.function_call.arguments,
context=validation_context,
strict=strict,
)
elif mode == Mode.TOOLS:
assert (
len(message.tool_calls) == 1
), "Instructor does not support multiple tool calls, use List[Model] instead."
tool_call = message.tool_calls[0]
assert (
tool_call.function.name == cls.openai_schema["name"] # type: ignore[index]
), "Tool name does not match"
return cls.model_validate_json(
tool_call.function.arguments,
context=validation_context,
strict=strict,
)
elif mode in {Mode.JSON, Mode.JSON_SCHEMA, Mode.MD_JSON}:
> return cls.model_validate_json(
message.content,
context=validation_context,
strict=strict,
)
E pydantic_core._pydantic_core.ValidationError: 1 validation error for ReportDetails
E Invalid JSON: EOF while parsing a value at line 1 column 0 [type=json_invalid, input_value='', input_type=str]
E For further information visit https://errors.pydantic.dev/2.6/v/json_invalid
Any suggestions would be most welcome.
Cheers
Since a couple of days I've been getting this error a lot of times, broking the system I was building. Any solution?
same even for me - Even with this dummy code
from openai import AzureOpenAI from pydantic import BaseModel
client = instructor.patch(AzureOpenAI( api_version="", azure_endpoint="", azure_deployment="" )) class UserDetail(BaseModel): name: str age: int
user = client.chat.completions.create( model="", response_model=UserDetail, messages=[ {"role": "user", "content": "Extract Jason is 25 years old"}, ] )
print(user)
nstructorRetryException: 1 validation error for UserDetail Invalid JSON: expected value at line 1 column 1 [type=json_invalid, input_value="const Jason: UserDetail ...e: 'Jason', age: 25 });", input_type=str]
pleaese use from_openai rather than patch. but what model did you use and why is the input javascript? const Jason: UserDetail ...e: 'Jason', age: 25 })
