storm icon indicating copy to clipboard operation
storm copied to clipboard

[Bug] Issue with o3-mini Model in AzureOpenAIModel Due to max_tokens Parameter

Open Junaid-Nazir-828 opened this issue 8 months ago • 0 comments

Description:

When using o3-mini instead of gpt-4o in AzureOpenAIModel, the API call fails due to the use of max_tokens instead of max_completion_tokens. o3-mini does not support max_tokens, leading to an error when making requests.

Reproduction Steps:

  1. Set up a Flask API that calls AzureOpenAIModel with o3-mini.
  2. Use max_tokens in the request payload instead of max_completion_tokens.
  3. Observe that the API returns an error indicating that max_tokens is unsupported.

Flask App Code:

from flask import Flask, request, jsonify
from knowledge_storm.lm import AzureOpenAIModel

app = Flask(__name__)

@app.route('/api/test', methods=['POST'])
def test_api():
    data = request.json
    query = data.get('query', 'Test query')
    max_tokens = data.get('max_tokens', 4000)
    
    # Initialize model
    llm = AzureOpenAIModel(
        model="o3-mini",
        api_key="your-api-key",
        azure_endpoint="your-endpoint",
        api_version="your-api-version",
        max_tokens=max_tokens  # This will cause an error with o3-mini
    )
    
    response = llm.basic_request(query)
    return jsonify(response)

if __name__ == "__main__":
    app.run(debug=True)

Expected Behavior:

  • AzureOpenAIModel should recognize when o3-mini is used and switch max_tokens to max_completion_tokens dynamically.

Actual Behavior:

  • The request fails with an error:
{
    "error": {
        "code": "unsupported_parameter",
        "message": "Unsupported parameter: 'max_tokens' is not supported with this model. Use 'max_completion_tokens' instead."
    }
}

Additional Context:

  • The issue arises because o3-mini uses a different parameter naming convention compared to gpt-4o.

Junaid-Nazir-828 avatar Mar 30 '25 23:03 Junaid-Nazir-828