[BUG] LLMConfig.max_tokens not respected by OpenAI-based clients
LLMConfig.max_tokens not respected by OpenAI-based clients
Bug Description
When configuring max_tokens in LLMConfig, the setting is ignored by OpenAI-based clients (OpenAIClient and AzureOpenAILLMClient), causing token limit errors even when a higher limit is configured.
Additionally, there are currently two confusing ways to set max_tokens for OpenAI clients:
- Through
LLMConfig(max_tokens=...) - Through the client constructor parameter
OpenAIClient(max_tokens=...)
This dual configuration is confusing because:
- It's unclear which one takes precedence
- Users naturally expect the config object to be the authoritative source
- The constructor parameter silently overrides the config setting
- Other clients (Gemini, Anthropic) don't have this confusing dual interface
What I was trying to do
Configure a higher token limit for LLM responses to avoid truncation:
from graphiti_core.llm_client.config import LLMConfig
from graphiti_core.llm_client.openai_client import OpenAIClient
config = LLMConfig(
api_key="...",
model="gpt-4",
max_tokens=16000 # Set higher limit
)
client = OpenAIClient(config=config)
What I expected to happen
The client should respect the max_tokens=16000 setting and allow responses up to 16,000 tokens, similar to how other configuration options work.
What actually happened
The client still uses the default 8192 token limit, resulting in errors like: Output length exceeded max tokens 8192: Could not parse response content as the length limit was reached
Root Cause
In BaseOpenAIClient.__init__(), the constructor parameter max_tokens=DEFAULT_MAX_TOKENS always overrides config.max_tokens, even when explicitly set by the user:
# Current problematic code:
def __init__(self, config: LLMConfig | None = None, cache: bool = False, max_tokens: int = DEFAULT_MAX_TOKENS):
# ...
super().__init__(config, cache)
self.max_tokens = max_tokens # This overwrites config.max_tokens!
Code sample that demonstrates the issue
from graphiti_core.llm_client.config import LLMConfig
from graphiti_core.llm_client.openai_client import OpenAIClient
config = LLMConfig(max_tokens=16000)
client = OpenAIClient(config=config)
print(f"Config max_tokens: {config.max_tokens}") # Shows: 16000
print(f"Client max_tokens: {client.max_tokens}") # Shows: 8192 (wrong!)
# User expects: client.max_tokens == 16000
# Actual result: client.max_tokens == 8192
Affected Clients
OpenAIClientAzureOpenAILLMClient
Other clients handle this correctly:
GeminiClient- respects config and has clear precedence rulesAnthropicClient- respects config settingsGroqClient- respects config settings
Proposed Solution
The fix should:
- Respect
config.max_tokenswhen explicitly set (different from default) - Use constructor parameter as fallback when config has default value
- Maintain backward compatibility for existing code
- Clear precedence rules: Config takes priority over constructor parameter
# Proposed fix:
if config.max_tokens != DEFAULT_MAX_TOKENS:
self.max_tokens = config.max_tokens # Use config when explicitly set
else:
self.max_tokens = max_tokens # Use constructor parameter as fallback
Environment
- Graphiti version: [current version]
- Python version: 3.12+
- Affected clients: OpenAIClient, AzureOpenAILLMClient
Labels
bughelp wantedgood first issuellm-client
Thanks for the PR. I've left some comments.
same! I still have this issue on the latest version. causing issues.
修改代码后仍然报相同的错
Hi - Any update on the fix for this getting merged in?
@paveljakov Is this still an issue? Please confirm within 14 days or this issue will be closed.
@paveljakov Is this still an issue? Please confirm within 14 days or this issue will be closed.