Cannot use Graphiti without OPEN AI KEY

[martin-san03]

Hello, I'm trying to setup a demo with graphiti using Gemeni, I don't have OpenAI Account neither the Key.

Even if I configure Graphiti as per the tutorial for using Gemeni:

graphiti = Graphiti(
    uri=neo4j_uri, 
    user=neo4j_user, 
    password=neo4j_password,
    llm_client=GeminiClient(
        config=LLMConfig(
            api_key=api_key,
            model="gemini-2.5-flash-preview-05-20"
        )
    ),
    embedder=GeminiEmbedder(
        config=GeminiEmbedderConfig(
            api_key=api_key,
            embedding_model="embedding-001"
        )
    ),

I have an error because I don´t have the OpenAPI key in my environment:

Exception has occurred: OpenAIError       (note: full exception trace is shown but execution is paused at: <module>)
The api_key client option must be set either by passing api_key to the client or by setting the OPENAI_API_KEY environment variable

I suppose is because of the default cross_encoder client, which is an OpenAIRerankerClient. Any suggestions to workaround of fix this issue?

[dipanjannC]

Hi,

I also had a similar doubt. I'm trying to use an API key from Groq along with a CustomCrossEncoder utilizing BGERerankerClient.

Custom CrossEncoder Implementation

class CustomCrossEncoderReranker(BGERerankerClient):
    def __init__(self, model_name: str = "cross-encoder/ms-marco-MiniLM-L-6-v2", custom_threshold: float = 0.1):
        super().__init__()
        self.custom_threshold = custom_threshold
        print(f"CustomCrossEncoderReranker initialized with model: '{model_name}' and custom threshold: {self.custom_threshold}")

    def rerank(self, query: str, documents: list[str]) -> list[tuple[str, float]]:
        if not documents:
            return []

        # Call parent's reranking logic
        original_reranked_tuples = super().rerank(query=query, documents=documents)

        custom_results = []
        print(f"🔎 Query for custom reranking: '{query}'")
        print(f"📝 Original reranked results (first 5): {original_reranked_tuples[:5]}")

        for doc, score in original_reranked_tuples:
            if score >= self.custom_threshold:
                adjusted_score = min(score * 1.05, 1.0)
                custom_results.append((doc, adjusted_score))

        custom_results.sort(key=lambda x: x[1], reverse=True)

        print(f"Custom reranking applied: Initial count {len(original_reranked_tuples)}, Final count {len(custom_results)}")
        print(f"Custom reranked results (first 5): {custom_results[:5]}")
        return custom_results

---

Initialization Code

# Initialize Groq LLM client
groq_client = GroqClient(
    config=LLMConfig(
        api_key=GROQ_API_KEY,
        max_tokens=DEFAULT_MAX_TOKENS,
        model="llama-3.1-8b-instant"
    )
)

# Initialize Gemini Embedder
embedder_config = GeminiEmbedderConfig(api_key=os.getenv("GEMINI_API_KEY"), embedding_model="embedding-001")
embedder = GeminiEmbedder(config=embedder_config)

---

Adding an Episode to the Knowledge Graph

async def add_episode(kg: KnowldegeGraph):
    await kg.graph.add_episode(
        name="tech_innovation_article",
        episode_body=(
            "MIT researchers have unveiled 'ClimateNet', an AI system capable of predicting "
            "climate patterns with unprecedented accuracy. Early tests show it can forecast "
            "major weather events up to three weeks in advance, potentially revolutionizing "
            "disaster preparedness and agricultural planning."
        ),
        source=EpisodeType.text,
        source_description="Technology magazine article",
        reference_time=datetime(2023, 11, 15, 9, 30),
    )
    return kg

---

However, this throws an error:

{
  "error": {
    "message": "Failed to generate JSON. Please adjust your prompt. See 'failed_generation' for more details.",
    "type": "invalid_request_error",
    "code": "json_validate_failed"
  }
}

---

Question

Am I doing anything wrong here?

• I have a KG class that initializes the graphiti graph.
• The add_episode() method seems to be failing due to JSON generation issues.
• Could the problem be with the episode_body length or special characters in the text?

Would appreciate any insights!

[martin-san03]

I have made one with help of gemeni and the OpenAI one:

import asyncio
import re
import os
from typing import List, Tuple, Optional

from google import genai  
from google.genai import types  

from pydantic import BaseModel, Field
from graphiti_core.cross_encoder.client import CrossEncoderClient


# Configuration for the Gemini Reranker
class GeminiRerankerConfig(BaseModel):
    api_key: Optional[str] = Field(None, description="Google API Key for Gemini.")
    model: str = Field("gemini-1.5-flash-latest", description="Gemini model to use for reranking.")

class GeminiRerankerClient(CrossEncoderClient):
    """
    A CrossEncoderClient implementation that uses a Gemini model to rerank passages.
    """

    def __init__(self, config: Optional[GeminiRerankerConfig] = None):
        if config is None:
            config = GeminiRerankerConfig()
        self.config = config

        if not self.config.api_key:
            # Fallback to environment variable if api_key is not in config,
            # though your RagService._init_graphiti passes it from config.
            # This line might not be strictly necessary if config always provides it.
            resolved_api_key = self.config.api_key or os.environ.get('GOOGLE_API_KEY')
            if not resolved_api_key:
                raise ValueError(
                    "API key must be provided for GeminiRerankerClient either in config"
                    " or as GOOGLE_API_KEY environment variable."
                )
        else:
            resolved_api_key = self.config.api_key

        # Configure the Gemini API
        self.client = genai.Client(
            api_key=config.api_key,
        ) 
        # Create generation config
        self.generation_config = types.GenerateContentConfig(
            temperature=0.0,
            #max_output_tokens=max_tokens or self.max_tokens,
            #response_mime_type='application/json' if response_model else None,
            #response_schema=response_model if response_model else None,
            #system_instruction=system_prompt,
        )


    def _construct_prompt(self, query: str, passage: str) -> str:
        """Constructs a prompt for the LLM to score passage relevance."""
        # Improved prompt for more reliable numerical output
        return (
            f"Query: \"{query}\"\n\n"
            f"Passage: \"{passage}\"\n\n"
            "Based on the query and the passage, how relevant is the passage to the query? "
            "Please provide a relevance score as a single floating-point number between 0.0 (not relevant at all) "
            "and 1.0 (highly relevant). IMPORTANT: Output ONLY the numerical score. For example: 0.75"
            "\n\nScore: "
        )

    async def _get_score_for_passage(self, query: str, passage: str) -> float:
        """Gets a relevance score for a single passage using the Gemini model."""
        prompt = self._construct_prompt(query, passage)
        try:
            # Generate content using the simple string approach
            response = await self.client.aio.models.generate_content(
                model=self.model,
                contents=[prompt],
                generation_config=self.generation_config
            )
            
            # Attempt to parse the score
            raw_text = response.text.strip()
            # Try to find a float in the response text
            match = re.search(r"[-+]?\d*\.\d+|\d+", raw_text) # Looks for float or integer
            if match:
                score = float(match.group(0))
                # Clamp score between 0.0 and 1.0
                return max(0.0, min(1.0, score))
            else:
                print(f"Warning: Could not parse score from Gemini response for passage. Response: '{raw_text}'. Assigning default score 0.0.")
                return 0.0
        except Exception as e:
            print(f"Error calling Gemini API for reranking passage: {e}. Assigning default score 0.0.")
            return 0.0

    async def rank(self, query: str, passages: List[str]) -> List[Tuple[str, float]]:
        """
        Rank the given passages based on their relevance to the query using Gemini.

        Args:
            query (str): The query string.
            passages (list[str]): A list of passages to rank.

        Returns:
            list[tuple[str, float]]: A list of tuples containing the passage and its score,
                                     sorted in descending order of relevance.
        """
        if not passages:
            return []

        scored_passages: List[Tuple[str, float]] = []

        # Create tasks for all passages to be scored concurrently
        tasks = [self._get_score_for_passage(query, passage) for passage in passages]
        scores = await asyncio.gather(*tasks)

        for passage, score in zip(passages, scores):
            scored_passages.append((passage, score))

        # Sort by score in descending order
        scored_passages.sort(key=lambda item: item[1], reverse=True)

        return scored_passages

I'm trying to run it now. I does initialize, but I'm having this issue now:

Traceback (most recent call last):
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\work\session.py", line 583, in _run_transaction
    result = await transaction_function(tx, *args, **kwargs)
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\driver.py", line 1307, in _work
    res = await tx.run(query, parameters)
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\work\transaction.py", line 206, in run
    await result._tx_ready_run(query, parameters)
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\work\result.py", line 177, in _tx_ready_run
    await self._run(query, parameters, None, None, None, None, None, None)
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\work\result.py", line 235, in _run
    await self._connection.send_all()
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\io\_common.py", line 195, in inner
    await coroutine_func(*args, **kwargs)
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\io\_bolt.py", line 835, in send_all
    await self._send_all()
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\io\_bolt.py", line 819, in _send_all
    if await self.outbox.flush():
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async\io\_common.py", line 144, in flush
    await self.socket.sendall(data)
  File "c:\Private\ai\crackit\backend\.venv\lib\site-packages\neo4j\_async_compat\network\_bolt_socket.py", line 161, in sendall
    self._writer.write(data)
  File "C:\Program Files\ANSYS Inc\v242\SCADE\contrib\Python310\lib\asyncio\streams.py", line 325, in write
    self._transport.write(data)
  File "C:\Program Files\ANSYS Inc\v242\SCADE\contrib\Python310\lib\asyncio\proactor_events.py", line 361, in write
    self._loop_writing(data=bytes(data))
  File "C:\Program Files\ANSYS Inc\v242\SCADE\contrib\Python310\lib\asyncio\proactor_events.py", line 397, in _loop_writing
    self._write_fut = self._loop._proactor.send(self._sock, data)
AttributeError: 'NoneType' object has no attribute 'send'

In one of all my trials, It did succeed to register episodes in the neo4j database:

It does not seems to be related. Any idea on how to debug this? Thanks in advance. Kind regads,

[martin-san03]

I believe I found the issue:

2025-06-03 13:06:28,782 - root - INFO - Updating RAG database with deleted BaseNode 863.
2025-06-03 13:06:30,888 - google_genai.models - INFO - AFC is enabled with max remote calls: 10.
2025-06-03 13:06:33,636 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-preview-05-20:generateContent "HTTP/1.1 200 OK"
2025-06-03 13:06:33,660 - google_genai.models - INFO - AFC is enabled with max remote calls: 10.
2025-06-03 13:06:33,976 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/embedding-001:batchEmbedContents "HTTP/1.1 200 OK"
2025-06-03 13:06:33,993 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/embedding-001:batchEmbedContents "HTTP/1.1 200 OK"
2025-06-03 13:06:36,075 - google_genai.models - INFO - AFC is enabled with max remote calls: 10.
2025-06-03 13:06:38,538 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-preview-05-20:generateContent "HTTP/1.1 200 OK"
2025-06-03 13:06:40,415 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-preview-05-20:generateContent "HTTP/1.1 200 OK"
2025-06-03 13:06:40,430 - google_genai.models - INFO - AFC is enabled with max remote calls: 10.
2025-06-03 13:06:40,444 - google_genai.models - INFO - AFC is enabled with max remote calls: 10.
2025-06-03 13:06:40,863 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/embedding-001:batchEmbedContents "HTTP/1.1 200 OK"
2025-06-03 13:06:40,923 - google_genai.models - INFO - AFC is enabled with max remote calls: 10.

and then:

2025-06-03 13:06:41,029 - httpx - INFO - HTTP Request: POST https://generativelanguage.googleapis.com/v1beta/models/gemini-2.5-flash-preview-05-20:generateContent "HTTP/1.1 429 Too Many Requests"

Could a Too Many Request response close the socket and cause the 'NoneType' object has no attribute 'send'? It may lack of some robustness checks somewhere...

[martin-san03]

I found the problem, I was using async_to_sync to convert async functions to sync and be able to call it inside my synchronous application.

Using

ev_loop = asyncio.get_event_loop()
ev_loop.run_until_complete(graphiti.add_episode(...))

instead solves the issue. Unfortunately, because how the app is built, I need to do so for each call to graphiti API.

I will test now the cross_encoder.

[Strophe27]

Had the same problem with openrouter, it's because openrouter api doesn't have embedding enpoint.. is it the same problem for you ?

[prasmussen15]

Hey, it looks like some of you are experiencing different issues here. There are three objects which use the OpenAI key: llm_client, embedder, and cross_encoder. You have to make sure you are passing values for all 3 if you are not using OpenAI.

Also the failed JSON structure output that @dipanjannC was experiencing is because the llama-3.1-8b was having difficulty following the structured output during extraction (which is common with some of the smaller models).