fastapi icon indicating copy to clipboard operation
fastapi copied to clipboard

[BUG] OpenAPI `KeyError` on body parameter when field types resolve to the same type

Open mayteio opened this issue 2 years ago • 7 comments

First Check

  • [X] I added a very descriptive title to this issue.
  • [X] I used the GitHub search to find a similar issue and didn't find it.
  • [X] I searched the FastAPI documentation, with the integrated search.
  • [X] I already searched in Google "How to X in FastAPI" and didn't find any information.
  • [X] I already read and followed all the tutorial in the docs and didn't find an answer.
  • [X] I already checked if it is not related to FastAPI but to Pydantic.
  • [X] I already checked if it is not related to FastAPI but to Swagger UI.
  • [X] I already checked if it is not related to FastAPI but to ReDoc.

Commit to Help

  • [X] I commit to help with one of those options 👆

Example Code

from enum import Enum
from fastapi import FastAPI
from pydantic import BaseModel
from typing import Generic, Optional, TypeVar

DataT = TypeVar("DataT")


class Type(str, Enum):
  green = "green"
  blue = "blue"


class State(str, Enum):
  published = "published"
  archived = "archived"


class FieldFilter(GenericModel, Generic[DataT]):
  eq: Optional[DataT]
  neq: Optional[DataT]

class StringFilter(FieldFilter[str]):
  ilike: Optional[str]
  nilike: Optional[str]

class Where(BaseModel):
  name: Optional[StringFilter]
  # changing either of these lines to FieldFilter[str] resolves the error
  # but obviously we'd like the enums to display in openapi.json rather than str
  type: Optional[FieldFilter[Type]]
  state: Optional[FieldFilter[State]]


class QueryParams(BaseModel):
  where: Optional[Where]
  limit: Optional[int]
  offset: Optional[int]


app = FastAPI()

@app.post("")
def search(params: QueryParams):
    return {"Hello":"Tiangolo"}

Description

  • Open the browser and hit /openapi.json
  • You see Internal Server Error when you should see the openapi.json
INFO:     172.31.0.1:62102 - "GET /openapi.json HTTP/1.1" 500 Internal Server Error
ERROR:    Exception in ASGI application
Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/uvicorn/protocols/http/httptools_impl.py", line 398, in run_asgi
    result = await app(self.scope, self.receive, self.send)
  File "/usr/local/lib/python3.9/site-packages/uvicorn/middleware/proxy_headers.py", line 45, in __call__
    return await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 199, in __call__
    await super().__call__(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/applications.py", line 111, in __call__
    await self.middleware_stack(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 181, in __call__
    raise exc from None
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/errors.py", line 159, in __call__
    await self.app(scope, receive, _send)
  File "/usr/local/lib/python3.9/site-packages/starlette/middleware/cors.py", line 78, in __call__
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/ddtrace/contrib/asgi/middleware.py", line 178, in __call__
    reraise(exc_type, exc_val, exc_tb)
  File "/usr/local/lib/python3.9/site-packages/six.py", line 719, in reraise
    raise value
  File "/usr/local/lib/python3.9/site-packages/ddtrace/contrib/asgi/middleware.py", line 173, in __call__
    return await self.app(scope, receive, wrapped_send)
  File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 82, in __call__
    raise exc from None
  File "/usr/local/lib/python3.9/site-packages/starlette/exceptions.py", line 71, in __call__
    await self.app(scope, receive, sender)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 566, in __call__
    await route.handle(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 227, in handle
    await self.app(scope, receive, send)
  File "/usr/local/lib/python3.9/site-packages/starlette/routing.py", line 41, in app
    response = await func(request)
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 152, in openapi
    return JSONResponse(self.openapi())
  File "/usr/local/lib/python3.9/site-packages/fastapi/applications.py", line 130, in openapi
    self.openapi_schema = get_openapi(
  File "/usr/local/lib/python3.9/site-packages/fastapi/openapi/utils.py", line 354, in get_openapi
    definitions = get_model_definitions(
  File "/usr/local/lib/python3.9/site-packages/fastapi/utils.py", line 28, in get_model_definitions
    model_name = model_name_map[model]
KeyError: <class 'app.gateways.rest.schemas.projects.FieldFilter[str]'>

Operating System

macOS

Operating System Details

No response

FastAPI Version

0.63.0

Python Version

3.9.7

Additional Context

Possibly related, but slightly different issues:

  • https://github.com/tiangolo/fastapi/issues/1505
  • https://github.com/tiangolo/fastapi/issues/2724 (seems stale)

Also, removing the str inheritance in either Type or State resolves the issue, though messes with the rest of our application logic.

mayteio avatar Nov 19 '21 00:11 mayteio

+1

jfitz1 avatar Jan 06 '22 05:01 jfitz1

+1

I am using Response[List[xx]] on many occasions. Getting a KeyError.

//edit: interestingly, this issue disappeared when downgraded from python3.10 to python3.8.

xeroc avatar May 09 '22 12:05 xeroc

+1

antoniodipinto avatar Jul 24 '22 18:07 antoniodipinto

@antoniodipinto I can no longer reproduce this on python 3.10 and FastAPI 0.79. Which versions are you using?

JarroVGIT avatar Jul 24 '22 18:07 JarroVGIT

Hi @JarroVGIT, seems that I have this issue on Python version 3.8 and FastAPI 0.79

antoniodipinto avatar Jul 24 '22 18:07 antoniodipinto

Weird. I just tried it with python 3.8.13 and FastAPI 0.79, and still couldn't reproduce. I took the code example from the issue opening post and ran it. When requesting /openapi.json, I am seeing a normal JSON and not this. You sure you have the same issue? I know there are similar issues like this one (also with KeyError) when using dataclasses rather than BaseModels. See this issue https://github.com/tiangolo/fastapi/issues/5138

JarroVGIT avatar Jul 24 '22 19:07 JarroVGIT

Update: everything is working fine right now. Probably some local python error. Reconfigured the venv and problem solved Thaks @JarroVGIT for the help

antoniodipinto avatar Aug 01 '22 15:08 antoniodipinto

Error Reason: fastapi converts a dataclasses.dataclass into class pydantic.BaseModel model. Thereby, fastapi treats the same dataclass model as a different class whenever a dataclass referenced mutliple times p.e. as response_model and at different endpoints (this behaviour is probably not intendet).

First, APIRoute constructor ćonverts the dataclasses as mentioned bevore. In this example, fastapi.utils.create_response_field called at fastapi.routing.APIRoute line 389 converts the response model for each route. create_response_field is called twice since both endpoints have the same response model. Hence, both endpoints do not have the exact same response model.

Next, the openapi schema is generated using fastapi.openapi.utils.get_openapi. This function calls pydantic.schema.get_model_name_map at line 413. get_model_name_map expects a set of unique models. In this case however, fastapi passes two different classes since id(route_one.response_model) != id(route_two.response_model), which share the same class name (and module name). Consequently, get_model_name_map removes one class from its result.

Finally, this ends up in a KeyError in get_model_definitions.

What to do until this bug is fixed? Basically there are three options:

  • Do not use the same model twice or avoid the openapi schema.
  • The bug occurs with pydantic v1.9.2 (current release) and before. However, pydantic's master branch has a rewritten version of pydantic.dataclasses that solves this issue.
  • Patch the errors during runtime. For instance, fastapi-xml uses a modified version of pydantic.dataclasses._process_class that fixex the issue. That patch solves also the pydantic issue #4353

cercide avatar Aug 18 '22 10:08 cercide