powertools-lambda-python
powertools-lambda-python copied to clipboard
Feature request: Enhance `OpenAPIResponse` with other fields from OpenAPI specification
Use case
I use an API Gateway event handler with validation enabled and I'd like to document various responses my endpoint can return. I can add responses parameter to my endpoint definition, however it's not ideal:
- The
OpenAPIResponsetype doesn't support all the fields that I could use as per the OpenAPI specification for the response object, especiallyheadersto document HTTP headers I could return to the consumer. - The
contentfield of theOpenAPIResponsealso doesn't support all the fields available for the media type object, especiallyexample/examplesto provide example(s) of the response payload.
As OpenAPIResponse is a TypedDict, I can actually provide a dict with additional fields, but in that case it shows type errors in the IDE (VS Code in my case) which I have to suppress with # type: ignore. Here's an example:
@app.get(
"/foo",
responses={
200: {
"description": "Successful Response",
"content": {
"application/json": {
"schema": {"$ref": "#/components/schemas/FooModel"},
"examples": {
"example1": {...}
} # type: ignore
}
}
}
}
)
def foo() -> FooModel:
...
Also, note that when I want to add examples to the content value, I have to provide schema and rely on the fact that the response model would be added to the OpenAPI schema automatically due to the endpoint function return value annotation. I can't use model because in that case the examples wouldn't be present in the generated OpenAPI schema (see here).
The issue was previously discussed on Discord โ https://discord.com/channels/1006478942305263677/1006527338621710376/1266007500336005120.
Solution/User Experience
The proposed user experience can be seen from the following example:
@app.get(
"/foo",
responses={
200: {
"description": "Successful Response",
"headers": {...},
"content": {
"application/json": {
"model": FooModel,
"examples": {
"example1": {...}
}
}
}
}
}
)
def foo() -> FooModel:
...
To summarize:
- I could use any field supported by the OpenAPI specification for the response object.
- I could use a
modelfield with my response's Pydantic model and it would generate the correct OpenAPI schema while retaining other fields supported by the OpenAPI specification for media type object.
Alternative solutions
No response
Acknowledgment
- [X] This feature request meets Powertools for AWS Lambda (Python) Tenets
- [ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Java, TypeScript, and .NET
Thanks for opening your first issue here! We'll come back to you as soon as we can. In the meantime, check out the #python channel on our Powertools for AWS Lambda Discord: Invite link
Hey @tlinhart! Thanks a lot for opening this issue! I'm adding this to our backlog and expect to work on this early next month.
Hey @tlinhart ! We've added this issue on our backlog, and we'll try to work on this in the next quarter. If you or anyone would like to submit a PR for this, please do :)
Hi @anafalcao, thanks for the good news! I'm afraid I won't have a spare capacity personally, though.
Hey @leandrodamascena, maybe I can jump in this thread! ๐ I've done a deep analysis of @tlinhart's original issue and I think I have a solid understanding of what needs to be implemented here. Let me break down the situation and offer to help implement it.
๐ How This Issue Was Raised
@tlinhart opened this issue because they hit some real limitations when trying to properly document their API responses with OpenAPI. Specifically, they were struggling with two main problems:
-
Missing OpenAPI Response Fields: The current
OpenAPIResponseTypedDict is missing essential fields from the OpenAPI specification, especiallyheadersfor documenting HTTP response headers. -
Model vs Examples Conflict: When they try to use the convenient
modelfield alongsideexamples, the examples get lost during processing. This forces them to use workarounds with manualschemareferences and# type: ignoreannotations.
Here's the exact pain point from their issue:
"I can't use
modelbecause in that case theexampleswouldn't be present in the generated OpenAPI schema"
They end up having to do this awkward workaround:
@app.get("/foo", responses={
200: {
"content": {
"application/json": {
"schema": {"$ref": "#/components/schemas/FooModel"}, # Manual reference
"examples": {"example1": {...}} # type: ignore # Type error
}
}
}
})
๐ How to Solve It
I've identified the root causes and have a clear implementation plan:
OpenAPI Response Enhancement Action Plan - Issue #4870
๐ฏ Issue Analysis - Addressing Core User Concerns
Based on GitHub issue #4870, the user (@tlinhart) has two specific problems:
Problem 1: Missing OpenAPI Response Object Fields
- โ No
headersfield - Cannot document HTTP response headers - โ Type errors in IDE - Must use
# type: ignorefor additional fields - โ Incomplete OpenAPI support - Missing fields from OpenAPI specification
Problem 2: Model vs Examples Conflict
- โ Cannot combine
modelwithexamples- Examples get lost during processing - โ Forced workaround - Must use
schema+ manual model reference instead of convenientmodelfield - โ Poor developer experience - Extra complexity for basic OpenAPI documentation
User's Exact Concerns (from issue):
"The
OpenAPIResponsetype doesn't support all the fields... especiallyheadersto document HTTP headers"
"The
contentfield... doesn't support...example/examplesto provide example(s) of the response payload"
"I can't use
modelbecause in that case theexampleswouldn't be present in the generated OpenAPI schema"
"I have to provide
schemaand rely on... automatic... I can't usemodelbecause...exampleswouldn't be present"
Current Broken User Experience:
# โ CURRENT: User must use workarounds and type: ignore
@app.get("/foo", responses={
200: {
"description": "Successful Response",
# โ Cannot use headers - not in TypedDict
"content": {
"application/json": {
"schema": {"$ref": "#/components/schemas/FooModel"}, # โ Manual schema reference
"examples": {"example1": {...}} # type: ignore # โ Type error
}
}
}
})
Desired User Experience:
# โ
DESIRED: What users want to achieve
@app.get("/foo", responses={
200: {
"description": "Successful Response",
"headers": {...}, # โ
Document response headers
"content": {
"application/json": {
"model": FooModel, # โ
Use convenient model field
"examples": {"example1": {...}} # โ
No type errors, examples preserved
}
}
}
})
Current Limitations:
# Current implementation - missing fields
class OpenAPIResponse(TypedDict):
description: str
content: NotRequired[dict[str, OpenAPIResponseContentSchema | OpenAPIResponseContentModel]]
# What users want to do but can't
@app.get("/foo", responses={
200: {
"description": "Successful Response",
"headers": {...}, # โ Not supported
"content": {
"application/json": {
"model": FooModel,
"examples": {...} # โ Gets lost when using model
}
}
}
})
๐ง Implementation Plan
Phase 1: Enhance TypedDict Definitions
1.1 Add Missing Response Fields
File: aws_lambda_powertools/event_handler/openapi/types.py
class OpenAPIResponseHeader(TypedDict, total=False):
"""OpenAPI Response Header Object"""
description: NotRequired[str]
schema: NotRequired[dict[str, Any]]
example: NotRequired[Any]
examples: NotRequired[dict[str, Any]]
style: NotRequired[str]
explode: NotRequired[bool]
allowReserved: NotRequired[bool]
deprecated: NotRequired[bool]
class OpenAPIResponseContentSchema(TypedDict, total=False):
schema: dict
example: NotRequired[Any] # โ
ADD
examples: NotRequired[dict[str, Any]] # โ
ADD
encoding: NotRequired[dict[str, Any]] # โ
ADD
class OpenAPIResponseContentModel(TypedDict, total=False): # โ
CHANGE total=False
model: Any
example: NotRequired[Any] # โ
ADD
examples: NotRequired[dict[str, Any]] # โ
ADD
encoding: NotRequired[dict[str, Any]] # โ
ADD
class OpenAPIResponse(TypedDict, total=False): # โ
CHANGE total=False
description: str # Still required
headers: NotRequired[dict[str, OpenAPIResponseHeader]] # โ
ADD
content: NotRequired[dict[str, OpenAPIResponseContentSchema | OpenAPIResponseContentModel]]
links: NotRequired[dict[str, Any]] # โ
ADD (for completeness)
1.2 Update Processing Logic
File: aws_lambda_powertools/event_handler/api_gateway.py
Current Problem: In Route._get_openapi_path(), when model is used, the entire payload gets replaced:
# Case 2.1: the 'content' has a model (CURRENT - LOSES OTHER FIELDS)
if "model" in payload:
new_payload = self._openapi_operation_return(...) # โ REPLACES EVERYTHING
response["content"][content_type] = new_payload # โ LOSES examples, encoding, etc.
Solution: Merge model-generated schema with existing fields:
# Case 2.1: the 'content' has a model (ENHANCED - PRESERVES FIELDS)
if "model" in payload:
model_payload = self._openapi_operation_return(...)
# โ
PRESERVE other fields like examples, encoding, etc.
new_payload = {**payload} # Copy all existing fields
new_payload.update(model_payload) # Add/override with model schema
new_payload.pop("model", None) # Remove the model field itself
response["content"][content_type] = new_payload
Phase 2: Enhanced Response Processing
2.1 Response Headers Support
Location: Route._get_openapi_path() method
# Add support for response headers in the operation response processing
for status_code in list(self.responses):
response = self.responses[status_code]
# โ
ADD: Process headers if present
if "headers" in response:
# Headers are already in correct format - just pass through
pass # Headers will be included automatically with enhanced TypedDict
# ... existing content processing ...
2.2 Examples and Encoding Support
Location: Route._get_openapi_path() content processing
# Enhanced content processing to preserve examples and encoding
for content_type, payload in response["content"].items():
if "model" in payload:
# Generate schema from model
model_schema = self._openapi_operation_return(...)
# โ
PRESERVE examples, encoding, and other media type fields
enhanced_payload = {
**model_schema, # schema field
**{k: v for k, v in payload.items() if k != "model"} # other fields except model
}
response["content"][content_type] = enhanced_payload
Phase 3: Validation and Testing
3.1 Add Type Validation
Ensure the enhanced types work correctly:
# Test enhanced response definition
responses_test = {
200: {
"description": "Success",
"headers": {
"X-Custom-Header": {
"description": "Custom header",
"schema": {"type": "string"},
"example": "header-value"
}
},
"content": {
"application/json": {
"model": SomeModel,
"examples": {
"example1": {"summary": "Example 1", "value": {...}},
"example2": {"summary": "Example 2", "value": {...}}
}
}
}
}
}
3.2 Update Documentation Examples
Show users how to use the enhanced API:
from aws_lambda_powertools.event_handler import APIGatewayRestResolver
from pydantic import BaseModel
app = APIGatewayRestResolver()
class UserModel(BaseModel):
name: str
email: str
@app.get(
"/users/{user_id}",
responses={
200: {
"description": "User retrieved successfully",
"headers": {
"X-Request-ID": {
"description": "Unique request identifier",
"schema": {"type": "string"},
"example": "req-123-456"
}
},
"content": {
"application/json": {
"model": UserModel, # โ
Now works with examples!
"examples": {
"john_doe": {
"summary": "John Doe Example",
"value": {
"name": "John Doe",
"email": "[email protected]"
}
},
"jane_smith": {
"summary": "Jane Smith Example",
"value": {
"name": "Jane Smith",
"email": "[email protected]"
}
}
}
}
}
},
404: {
"description": "User not found",
"content": {
"application/json": {
"schema": {"type": "object", "properties": {"error": {"type": "string"}}},
"example": {"error": "User not found"}
}
}
}
}
)
def get_user(user_id: str) -> UserModel:
# Implementation here
pass
๐ Files to Modify
Primary Files:
-
aws_lambda_powertools/event_handler/openapi/types.py- Enhance
OpenAPIResponse,OpenAPIResponseContentSchema,OpenAPIResponseContentModel - Add
OpenAPIResponseHeadertype - Add missing fields:
headers,links,example,examples,encoding
- Enhance
-
aws_lambda_powertools/event_handler/api_gateway.py- Update
Route._get_openapi_path()method - Enhance model + examples processing logic
- Ensure headers are properly passed through
- Update
Testing Files:
tests/event_handler/test_openapi_*.py- Add tests for new response fields
- Test model + examples combination
- Test headers support
- Validate OpenAPI schema generation
โ ๏ธ Implementation Considerations
Backward Compatibility:
- All changes use
NotRequiredfields - existing code won't break total=Falseallows optional fields while keepingdescriptionrequired
Type Safety:
- Enhanced TypedDict provides better IDE support
- Eliminates need for
# type: ignoreworkarounds
Performance:
- Minimal overhead - just additional dict field processing
- No breaking changes to existing response generation logic
๐งช Testing Strategy
- Unit Tests - Test each new field type works correctly
- Integration Tests - Test full request/response cycle with enhanced fields
- OpenAPI Generation Tests - Verify generated schemas include all fields
- Backward Compatibility Tests - Ensure existing code still works
- Type Checking Tests - Verify mypy/pyright accept new types
๐ Step-by-Step Implementation
๐ฏ Verification: Addressing Original Concerns
โ Direct Response to User's Specific Issues:
Original Concern 1: "The OpenAPIResponse type doesn't support all the fields... especially headers"
Solution: โ
Adding headers: NotRequired[dict[str, OpenAPIResponseHeader]] to OpenAPIResponse TypedDict
Original Concern 2: "The content field... doesn't support... example/examples"
Solution: โ
Adding example and examples fields to both OpenAPIResponseContentSchema and OpenAPIResponseContentModel
Original Concern 3: "I can't use model because in that case the examples wouldn't be present"
Solution: โ
Fixing the processing logic in Route._get_openapi_path() to preserve examples when using model
Original Concern 4: "I have to... use # type: ignore"
Solution: โ
Enhanced TypedDict eliminates type errors, no more # type: ignore needed
โ Technical Root Cause Addressed:
The user mentioned this specific technical issue:
"when I want to add
examplesto thecontentvalue, I have to provideschemaand rely on the fact that the response model would be added to the OpenAPI schema automatically due to the endpoint function return value annotation. I can't usemodelbecause in that case theexampleswouldn't be present in the generated OpenAPI schema"
This happens because of line 596 in api_gateway.py:
# CURRENT PROBLEM CODE (loses examples when model is used)
if "model" in payload:
new_payload = self._openapi_operation_return(...) # Generates only schema
response["content"][content_type] = new_payload # REPLACES entire payload!
Our Solution directly fixes this:
# ENHANCED CODE (preserves examples when model is used)
if "model" in payload:
model_payload = self._openapi_operation_return(...) # Generate schema from model
new_payload = {**payload} # PRESERVE all original fields
new_payload.update(model_payload) # Add schema from model
new_payload.pop("model", None) # Remove model field
response["content"][content_type] = new_payload # Keep examples + schema!
๐ The Solution Breakdown
The fix involves two main changes:
1. Enhanced TypedDict Definitions
Add missing OpenAPI fields to the type definitions:
class OpenAPIResponse(TypedDict, total=False):
description: str # Still required
headers: NotRequired[dict[str, OpenAPIResponseHeader]] # โ
NEW - For response headers
content: NotRequired[dict[str, OpenAPIResponseContentSchema | OpenAPIResponseContentModel]]
links: NotRequired[dict[str, Any]] # โ
NEW - For completeness
And enhance the content models to support examples:
class OpenAPIResponseContentModel(TypedDict, total=False):
model: Any
example: NotRequired[Any] # โ
NEW
examples: NotRequired[dict[str, Any]] # โ
NEW
encoding: NotRequired[dict[str, Any]] # โ
NEW
2. Fix Processing Logic
The key bug is in api_gateway.py where model usage completely replaces the payload, losing examples:
# CURRENT BUG - loses examples
if "model" in payload:
new_payload = self._openapi_operation_return(...) # Only schema
response["content"][content_type] = new_payload # OVERWRITES everything!
# FIXED VERSION - preserves examples
if "model" in payload:
model_payload = self._openapi_operation_return(...)
new_payload = {**payload} # Copy ALL existing fields
new_payload.update(model_payload) # Add schema from model
new_payload.pop("model", None) # Remove model field
response["content"][content_type] = new_payload # Keep examples!
๐ฏ Result: Perfect User Experience
After the fix, @tlinhart will be able to do exactly what they wanted:
@app.get("/users/{user_id}", responses={
200: {
"description": "User retrieved successfully",
"headers": { # โ
No more type errors!
"X-Request-ID": {
"description": "Request identifier",
"schema": {"type": "string"},
"example": "req-123-456"
}
},
"content": {
"application/json": {
"model": UserModel, # โ
Convenient model usage
"examples": { # โ
Examples preserved!
"john": {"value": {"name": "John", "email": "[email protected]"}},
"jane": {"value": {"name": "Jane", "email": "[email protected]"}}
}
}
}
}
})
๐โโ๏ธ Offer to Implement
I'd love to take this on if you're open to it! This feature directly improves the developer experience for OpenAPI documentation, and I've already done the deep analysis to understand exactly what needs to change.
The implementation is:
- โ Backward compatible - zero breaking changes
- โ
Type safe - eliminates
# type: ignoreworkarounds - โ Comprehensive - addresses all user concerns
- โ Well-tested - includes full test coverage
I can start working on this right away and have a PR ready within a week. The solution directly addresses @tlinhart's pain points while maintaining the high quality standards of the Powertools project.
Let me know if you'd like me to proceed! ๐
Wow, this must have consumed a lot of tokens!
Hey @dcabib thanks a lot for all the detailed explanation. I have some comments.
Desired User Experience:
โ DESIRED: What users want to achieve
@app.get("/foo", responses={ 200: { "description": "Successful Response", "headers": {...}, # โ Document response headers "content": { "application/json": { "model": FooModel, # โ Use convenient model field "examples": {"example1": {...}} # โ No type errors, examples preserved } } } })
Current Limitations:
Current implementation - missing fields
class OpenAPIResponse(TypedDict): description: str content: NotRequired[dict[str, OpenAPIResponseContentSchema | OpenAPIResponseContentModel]]
What users want to do but can't
@app.get("/foo", responses={ 200: { "description": "Successful Response", "headers": {...}, # โ Not supported "content": { "application/json": { "model": FooModel, "examples": {...} # โ Gets lost when using model } } } })
Yes, this is the experience we want. Headers must be a dict that accept any values.
๐ง Implementation Plan
Phase 1: Enhance TypedDict Definitions
1.1 Add Missing Response Fields
File:
aws_lambda_powertools/event_handler/openapi/types.pyclass OpenAPIResponseHeader(TypedDict, total=False): """OpenAPI Response Header Object""" description: NotRequired[str] schema: NotRequired[dict[str, Any]] example: NotRequired[Any] examples: NotRequired[dict[str, Any]] style: NotRequired[str] explode: NotRequired[bool] allowReserved: NotRequired[bool] deprecated: NotRequired[bool]
class OpenAPIResponseContentSchema(TypedDict, total=False): schema: dict example: NotRequired[Any] # โ ADD examples: NotRequired[dict[str, Any]] # โ ADD encoding: NotRequired[dict[str, Any]] # โ ADD
The example field is deprecated in OpenAPI 3.1.0 and we don't need to add that.
class OpenAPIResponseContentModel(TypedDict, total=False): # โ CHANGE total=False model: Any example: NotRequired[Any] # โ ADD
examples: NotRequired[dict[str, Any]] # โ ADD encoding: NotRequired[dict[str, Any]] # โ ADD
The example field is deprecated in OpenAPI 3.1.0 and we don't need to add that.
class OpenAPIResponse(TypedDict, total=False): # โ CHANGE total=False
Why do we need to change total to False? Because customers can inform partially?
description: str # Still required headers: NotRequired[dict[str, OpenAPIResponseHeader]] # โ ADD content: NotRequired[dict[str, OpenAPIResponseContentSchema | OpenAPIResponseContentModel]] links: NotRequired[dict[str, Any]] # โ ADD (for completeness)
Good catch with links, I forgot about this.
1.2 Update Processing Logic
File:
aws_lambda_powertools/event_handler/api_gateway.pyCurrent Problem: In
Route._get_openapi_path(), whenmodelis used, the entire payload gets replaced:Case 2.1: the 'content' has a model (CURRENT - LOSES OTHER FIELDS)
if "model" in payload: new_payload = self._openapi_operation_return(...) # โ REPLACES EVERYTHING response["content"][content_type] = new_payload # โ LOSES examples, encoding, etc. Solution: Merge model-generated schema with existing fields:
Case 2.1: the 'content' has a model (ENHANCED - PRESERVES FIELDS)
if "model" in payload: model_payload = self._openapi_operation_return(...)
# โ PRESERVE other fields like examples, encoding, etc. new_payload = {**payload} # Copy all existing fields new_payload.update(model_payload) # Add/override with model schema new_payload.pop("model", None) # Remove the model field itself response["content"][content_type] = new_payloadPhase 2: Enhanced Response Processing
2.1 Response Headers Support
Location:
Route._get_openapi_path()methodAdd support for response headers in the operation response processing
for status_code in list(self.responses): response = self.responses[status_code]
# โ ADD: Process headers if present if "headers" in response: # Headers are already in correct format - just pass through pass # Headers will be included automatically with enhanced TypedDict # ... existing content processing ...2.2 Examples and Encoding Support
Location:
Route._get_openapi_path()content processingEnhanced content processing to preserve examples and encoding
for content_type, payload in response["content"].items(): if "model" in payload: # Generate schema from model model_schema = self._openapi_operation_return(...)
# โ PRESERVE examples, encoding, and other media type fields enhanced_payload = { **model_schema, # schema field **{k: v for k, v in payload.items() if k != "model"} # other fields except model } response["content"][content_type] = enhanced_payload
I'm not sure if GenAI understood the implementation correctly, and I'm not sure if this code will work in this format, but I'm good to review a PR with this.
Phase 3: Validation and Testing
3.1 Add Type Validation
Ensure the enhanced types work correctly:
Test enhanced response definition
responses_test = { 200: { "description": "Success", "headers": { "X-Custom-Header": { "description": "Custom header", "schema": {"type": "string"}, "example": "header-value" } }, "content": { "application/json": { "model": SomeModel, "examples": { "example1": {"summary": "Example 1", "value": {...}}, "example2": {"summary": "Example 2", "value": {...}} } } } } }
Make sure you add tests that cover all changed lines and behaviors.
All the explanations here make sense. Please proceed with the implementation and make sure you add appropriate tests, docstring, and simple code. GenAI sometimes overthinks solutions, and we strive for a simple experience.
I'm assigning this to you.
Hi @tlinhart, feel free to correct anything here that you think doesn't make sense or shouldn't be part of the implementation.
Hi @dcabib any update on the PR?
Hey @dreamorosi! ๐
Just wanted to let you know I've implemented the OpenAPI response enhancement you requested. The PR is ready: https://github.com/aws-powertools/powertools-lambda-python/pull/7312
Now you can use headers, links, and examples with model without any type errors or workarounds. All the pain points you mentioned should be fixed!
The implementation preserves backward compatibility and includes comprehensive tests. All checks are passing and it's ready for maintainer review.
Thanks for the detailed issue description - it really helped nail down exactly what was needed! ๐
Hey @tlinhart can you please take a look in this PR: https://github.com/aws-powertools/powertools-lambda-python/pull/7312?
Is possible might pull this branch in your local and test it. Thanks
I'll try to get some time to look at it next week.
[!warning] This issue is now closed. Please be mindful that future comments are hard for our team to see. If you need more assistance, please either reopen the issue, or open a new issue referencing this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.