graphiti icon indicating copy to clipboard operation
graphiti copied to clipboard
verified publisher icon getzep

Question: How to setup Graphiti and Azure OpenAI?

Open dainius-s opened this issue 7 months ago • 1 comments

I really struggling to get this setup working with anything really. My biggest issues is that I have never used Azure (and I though AWS UI was bad...).

AZURE

API URL

Apart from that there are way too many options for the API endpoints on Azure and I have no clue what the api url format should look like in the firts place. It would be great if in the docs at least would give an example of the URL format and then there is also trailing slash, do I need need (don't need it)?

Embedings

As a super novice, I am still stuggling to understand how does all of this comes together? Does the embeddings have to be deployed onder same region in the Azure? From my exprience it was really hard to find a region that had both ChatGPT and Embedings available in the same region.

Azure Config Page

here is the link where I got my Azure config from is this is correct page?

Environment example.

Here is my env example, do I even configure it right? Is my Azure OpenAI URI correct format?

# Neo4j Database Configuration
# These settings are used to connect to your Neo4j database
NEO4J_URI=bolt://localhost:7687
NEO4J_USER=neo4j
NEO4J_PASSWORD=demodemo

# OpenAI API Configuration
# Required for LLM operations
OPENAI_API_KEY=XXXXXXXXXXXXXXXXXXXX
MODEL_NAME=gpt-4o-mini

# Azure OpenAI configuration
# Optional: Only needed for Azure OpenAI endpoints
AZURE_OPENAI_ENDPOINT=https://XXXXXXXXXX.openai.azure.com
AZURE_OPENAI_API_VERSION=2025-03-01-preview
AZURE_OPENAI_DEPLOYMENT_NAME=gpt-4o-mini
AZURE_OPENAI_EMBEDDING_API_VERSION=2023-05-15
AZURE_OPENAI_EMBEDDING_DEPLOYMENT_NAME=text-embedding-3-small
AZURE_OPENAI_USE_MANAGED_IDENTITY=false

When I launch docker I get these errors in the Graphiti service

2025-05-27 16:58:44 2025-05-27 15:58:44,964 - __main__ - INFO - Generated random group_id: default
2025-05-27 16:58:44 2025-05-27 15:58:44,964 - __main__ - INFO - Entity extraction disabled (no custom entities will be used)
2025-05-27 16:58:45 2025-05-27 15:58:45,083 - __main__ - ERROR - Failed to initialize Graphiti: 2 validation errors for GraphitiClients
2025-05-27 16:58:45 llm_client
2025-05-27 16:58:45   Input should be an instance of LLMClient [type=is_instance_of, input_value=<openai.lib.azure.AsyncAz...bject at 0x7efdbe58e490>, input_type=AsyncAzureOpenAI]
2025-05-27 16:58:45     For further information visit https://errors.pydantic.dev/2.11/v/is_instance_of
2025-05-27 16:58:45 embedder
2025-05-27 16:58:45   Input should be an instance of EmbedderClient [type=is_instance_of, input_value=<openai.lib.azure.AsyncAz...bject at 0x7efdbe59db10>, input_type=AsyncAzureOpenAI]
2025-05-27 16:58:45     For further information visit https://errors.pydantic.dev/2.11/v/is_instance_of
2025-05-27 16:58:45 2025-05-27 15:58:45,083 - __main__ - ERROR - Error initializing Graphiti MCP server: 2 validation errors for GraphitiClients
2025-05-27 16:58:45 llm_client
2025-05-27 16:58:45   Input should be an instance of LLMClient [type=is_instance_of, input_value=<openai.lib.azure.AsyncAz...bject at 0x7efdbe58e490>, input_type=AsyncAzureOpenAI]
2025-05-27 16:58:45     For further information visit https://errors.pydantic.dev/2.11/v/is_instance_of
2025-05-27 16:58:45 embedder
2025-05-27 16:58:45   Input should be an instance of EmbedderClient [type=is_instance_of, input_value=<openai.lib.azure.AsyncAz...bject at 0x7efdbe59db10>, input_type=AsyncAzureOpenAI]
2025-05-27 16:58:45     For further information visit https://errors.pydantic.dev/2.11/v/is_instance_of
2025-05-27 16:58:45 Traceback (most recent call last):
2025-05-27 16:58:45   File "/app/graphiti_mcp_server.py", line 1221, in <module>
2025-05-27 16:58:45     main()
2025-05-27 16:58:45   File "/app/graphiti_mcp_server.py", line 1214, in main
2025-05-27 16:58:45     asyncio.run(run_mcp_server())
2025-05-27 16:58:45   File "/usr/local/lib/python3.11/asyncio/runners.py", line 190, in run
2025-05-27 16:58:45     return runner.run(main)
2025-05-27 16:58:45            ^^^^^^^^^^^^^^^^
2025-05-27 16:58:45   File "/usr/local/lib/python3.11/asyncio/runners.py", line 118, in run
2025-05-27 16:58:45     return self._loop.run_until_complete(task)
2025-05-27 16:58:45            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-05-27 16:58:45   File "/usr/local/lib/python3.11/asyncio/base_events.py", line 654, in run_until_complete
2025-05-27 16:58:45     return future.result()
2025-05-27 16:58:45            ^^^^^^^^^^^^^^^
2025-05-27 16:58:45   File "/app/graphiti_mcp_server.py", line 1197, in run_mcp_server
2025-05-27 16:58:45     mcp_config = await initialize_server()
2025-05-27 16:58:45                  ^^^^^^^^^^^^^^^^^^^^^^^^^
2025-05-27 16:58:45   File "/app/graphiti_mcp_server.py", line 1188, in initialize_server
2025-05-27 16:58:45     await initialize_graphiti()
2025-05-27 16:58:45   File "/app/graphiti_mcp_server.py", line 576, in initialize_graphiti
2025-05-27 16:58:45     graphiti_client = Graphiti(
2025-05-27 16:58:45                       ^^^^^^^^^
2025-05-27 16:58:45   File "/app/.venv/lib/python3.11/site-packages/graphiti_core/graphiti.py", line 156, in __init__
2025-05-27 16:58:45     self.clients = GraphitiClients(
2025-05-27 16:58:45                    ^^^^^^^^^^^^^^^^
2025-05-27 16:58:45   File "/app/.venv/lib/python3.11/site-packages/pydantic/main.py", line 253, in __init__
2025-05-27 16:58:45     validated_self = self.__pydantic_validator__.validate_python(data, self_instance=self)
2025-05-27 16:58:45                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
2025-05-27 16:58:45 pydantic_core._pydantic_core.ValidationError: 2 validation errors for GraphitiClients
2025-05-27 16:58:45 llm_client
2025-05-27 16:58:45   Input should be an instance of LLMClient [type=is_instance_of, input_value=<openai.lib.azure.AsyncAz...bject at 0x7efdbe58e490>, input_type=AsyncAzureOpenAI]
2025-05-27 16:58:45     For further information visit https://errors.pydantic.dev/2.11/v/is_instance_of
2025-05-27 16:58:45 embedder
2025-05-27 16:58:45   Input should be an instance of EmbedderClient [type=is_instance_of, input_value=<openai.lib.azure.AsyncAz...bject at 0x7efdbe59db10>, input_type=AsyncAzureOpenAI]
2025-05-27 16:58:45     For further information visit https://errors.pydantic.dev/2.11/v/is_instance_of

Requests

I think it would be great to add some guidance to some new users trying setup a service as piecing things that are not straighforwards without much prior usage of some of the tech can be realy frustrating exprience! Any help would be greatly appreciated.

dainius-s avatar May 27 '25 16:05 dainius-s

Related issue here. Not clear at all how to provide successfully the Azure OpenAI Embedding Model configuration to initialize Graphitii:

Here is my quickstart.py

    # Initialize Graphiti with Azure OpenAI clients
    graphiti = Graphiti(
        neo4j_uri,
        neo4j_user,
        neo4j_password,
        llm_client=OpenAIClient(
            client=azure_openai_client
        ),
        embedder=OpenAIEmbedder(
            config=OpenAIEmbedderConfig(
                embedding_model="text-embedding-3-large"  # Use your Azure deployed embedding model name
            ),
            client=azure_openai_client
        ),
        cross_encoder=OpenAIRerankerClient(
            client=azure_openai_client
        )
    )

Errors occur when making the API requests:

Source Description: podcast transcript
2025-05-30 21:00:13 - httpx - INFO - HTTP Request: POST https://my-domain.openai.azure.com/openai/deployments/gpt-4o/chat/completions?api-version=2025-01-01-preview "HTTP/1.1 200 OK"
2025-05-30 21:00:13 - httpx - INFO - HTTP Request: POST https://my-domainopenai.azure.com/openai/deployments/gpt-4o/embeddings?api-version=2025-01-01-preview "HTTP/1.1 400 Bad Request"
2025-05-30 21:00:13 - httpx - INFO - HTTP Request: POST https://ai-mxsstr937477ai464760080861.openai.azure.com/openai/deployments/gpt-4o/embeddings?api-version=2025-01-01-preview "HTTP/1.1 400 Bad Request"
2025-05-30 21:00:13 - httpx - INFO - HTTP Request: POST https://ai-mxsstr937477ai464760080861.openai.azure.com/openai/deployments/gpt-4o/embeddings?api-version=2025-01-01-preview "HTTP/1.1 400 Bad Request"

Currently, its calling the wrong Model from my Azure deployments (gpt-4o instead of the "text-embedding-3-large") which is definitely deployed in my instance.

It would be helpful to have a detailed guideline for configuration with Azure OpenAI models (inference and embeddings) that works.

maxruby avatar May 30 '25 19:05 maxruby

@maxruby I ran into the same issue. The Graphiti documentation about Azure OpenAI is currently not correct.

The problem is related to using the same Azure OpenAI client for both the LLM and the embeddings actions. That leads to the problem that Graphiti is falsely trying to use the configured LLM deployment for embeddings, which of course fails.

The solution is to create and use two different Azure OpenAI clients, one for the LLM and one for the embeddings. Like so:

    # Azure OpenAI
    api_key = os.environ["AZURE_OPENAI_API_KEY"]
    api_version = os.environ["AZURE_OPENAI_API_VERSION"]
    azure_endpoint = os.environ["AZURE_OPENAI_ENDPOINT"]
    llm_deployment = os.environ["AZURE_OPENAI_DEPLOYMENT_NAME"]
    embedding_deployment = os.environ["AZURE_OPENAI_EMBEDDING_DEPLOYMENT"]

    # Initialize separate Azure OpenAI clients
    # LLM client for chat/completions
    azure_llm_client = AsyncAzureOpenAI(
        api_key=api_key,
        api_version=api_version,
        azure_endpoint=azure_endpoint,
        azure_deployment=llm_deployment,
    )

    # Embedding client for embeddings deployment
    azure_embed_client = AsyncAzureOpenAI(
        api_key=api_key,
        api_version=api_version,
        azure_endpoint=azure_endpoint,
        azure_deployment=embedding_deployment,
    )

    # Set up Graphiti
    graphiti = Graphiti(
        neo4j_uri,
        neo4j_user,
        neo4j_password,
        llm_client=OpenAIClient(
            client=azure_llm_client,
        ),
        embedder=OpenAIEmbedder(
            config=OpenAIEmbedderConfig(
                embedding_model=embedding_deployment,  # Your *embedding* model deployment!
            ),
            client=azure_embed_client,  # using the embedding deployment client
        ),
        cross_encoder=OpenAIRerankerClient(
            client=azure_llm_client,
        )
    )

dareima avatar Jun 09 '25 17:06 dareima

We use https://github.com/getzep/graphiti/pull/558/files in prod it works well

PrettyWood avatar Jun 25 '25 20:06 PrettyWood

This is described in the #558 kindly contributed by @PrettyWood

danielchalef avatar Jun 27 '25 03:06 danielchalef

@danielchalef the update regarding the use of Azure OpenAI in https://github.com/getzep/graphiti/blob/main/README.md is incorrect and not how @PrettyWood originally referenced his LLMConfig example.

OpenAIClient requires the argument config and doesn't support llm_config, which will lead to an error. Same goes for OpenAIRerankerClient.

Also, generally speaking, while the usage of one client with LLMConfig might work for ingesting facts into Graphiti / Neo4j, it doesn't for retrieving information from Graphiti via an AI chatbot using search recipes like NODE_HYBRID_SEARCH_CROSS_ENCODER.

At least for me it didn't in all my tests. There seem to be mixups in endpoint urls calling the proper model deployments and I got 404 errors. I reverted back to my solution with using two different clients for LLM and embeddings.

I am not a developer though and might be wrong with saying that.

dareima avatar Jun 27 '25 18:06 dareima

You're correct. Our recommendation is to use two different clients. I'll update the docs. Thank you.

danielchalef avatar Jun 27 '25 19:06 danielchalef