[BUG] "Failed to parse structured response" when adding episode using Gemini

[vanlinhtruongdang]

🐞 Bug Description

I'm building a Vietnamese knowledge base system for a chatbot using the following pipeline:

• doclayout YOLO to detect layout structure.

• Tesseract OCR to extract text.

• The extracted text is then pushed into Graphiti.

• Model configuration:

  • LLM base: gemini-2.5-flash-lite
  • Reranker: gemini-2.5-flash-lite
  • Embedder: gemini-embedding-001

However, starting July 31, 2025, I’ve been encountering persistent errors when trying to add an episode.

❗️ Error Message

Error in generating LLM response: Failed to parse structured response: Expecting value: line 1 column 1 (char 0)

Additionally:

• The system occasionally hangs for a while before crashing entirely.
• This issue did not occur before July 31, 2025 using the exact same configuration and input format.

✅ Things I’ve Verified

• OCR input is consistent and in Vietnamese as usual.
• No changes were made to the Graphiti setup or document processing pipeline.
• The LLM settings (gemini-2.5-flash-lite, gemini-embedding-001) remain unchanged from when it previously worked without errors.

🧪 Request

Please help investigate whether there were any internal changes or regressions to:

• The Gemini API (LLM base or embedding)
• Graphiti’s structured response parsing
• Rate limiting or timeout behaviors introduced recently

This issue is blocking the ingestion process and knowledge base construction.

Let me know if any logs or additional data are needed.

Thank you!

[matthiaslau]

I think this is because Gemini is answering with a json markdown block wrapped in "```json" + "```". But this should not happen as we set the response_mime_type here: https://github.com/getzep/graphiti/blob/506bc4fc984a55f8c1ac7845732570067288c837/graphiti_core/llm_client/gemini_client.py#L294

[matthiaslau]

It is working in version 0.14 but not working anymore in 0.15.1

[matthiaslau]

With 0.15 there was a small model introduced with a default to gemini-2.5-flash-lite-preview-06-17 and Flash Lite has a lot of problems regarding the json behaviour. For me this could be fixed by setting the small_model to something different for now, e.g.

llm_client=GeminiClient(
    config=LLMConfig(
        api_key=gemini_api_key,
        model="gemini-2.5-flash",
        small_model="gemini-2.5-flash",
    )
)

[claude[bot]]

@vanlinhtruongdang Is this still an issue? Please confirm within 14 days or this issue will be closed.

[hect1c]

I still get this issue, has anyone found a workaround?

[claude[bot]]

@vanlinhtruongdang Is this still an issue? Please confirm within 14 days or this issue will be closed.