datamodel-code-generator icon indicating copy to clipboard operation
datamodel-code-generator copied to clipboard

oneOf makes unparsable models and has inconsistent behavior for arrays

Open luminoso opened this issue 2 years ago • 0 comments

Describe the bug

I've encountered two different problems when trying to use datamode-code-generator with json schemas that have the oneOf property:

  1. oneOf generates code that can't parse valid data even in simple scenarios
  2. When oneOf is inside an array, there are properties missing from the generated classes

To Reproduce

For clarity, I'm going to use oneOf example from Liquid Studio.

From that example I've added additionalProperties: false to all the fields.

The resulting schema is the following:

{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "title": "My schema",
    "additionalProperties": false,
    "properties": {
        "AddressLine1": { "type": "string" },
        "AddressLine2": { "type": "string" },
        "City":         { "type": "string" }
    },
    "required": [ "AddressLine1" ],
    "oneOf": [
        {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "State":   { "type": "string" },
                "ZipCode": { "type": "string" }
            },
            "required": [ "ZipCode" ]
        },
        {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "County":   { "type": "string" },
                "PostCode": { "type": "string" }
            },
            "required": [ "PostCode" ]
        }
    ]
}

Which successfully validates, for example, the following json data:

{
    "AddressLine1" : "Microsoft Corporation",
    "AddressLine2" : "One Microsoft Way",
    "City" : "Redmond",
    "State" : "WA",
    "ZipCode" : "98052-7329"
}

When datamodel-code-generator parses the schema, the result is the following:

from __future__ import annotations

from typing import Optional, Union

from pydantic import BaseModel, Extra, Field


class Location1(BaseModel):
    class Config:
        extra = Extra.forbid

    AddressLine1: str
    AddressLine2: Optional[str] = None
    City: Optional[str] = None


class LocationItem(BaseModel):
    class Config:
        extra = Extra.forbid

    State: Optional[str] = None
    ZipCode: str


class LocationItem1(BaseModel):
    class Config:
        extra = Extra.forbid

    County: Optional[str] = None
    PostCode: str


class Location(BaseModel):
    class Config:
        extra = Extra.forbid

    __root__: Union[Location1, LocationItem, LocationItem1] = Field(..., title="My schema")

Which is almost correct: we can see that AddressLine1, AddressLine2 and City are there, and then State + ZipCode and County + PostCode in different models.

However, the data is unparsable, which is the first problem I've had when using schemas with oneOf (problem1). If we try to load a valid json payload, like so:

loc = Location.parse_obj(
    {
        "AddressLine1": "Microsoft Corporation",
        "AddressLine2": "One Microsoft Way",
        "City": "Redmond",
        "PostCode": "WA",
        "County": "98052-7329",
    }
)

pydantic will generate an exception. This is expected because none of the generated models can parse one complete json payload.

There's also a second problem.

This time let's encapsulate the first schema object example in an array:

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "array",
  "title": "My schema",
  "additionalProperties": false,
  "items": {
    "type": "object",
    "additionalProperties": false,
    "properties": {
        "AddressLine1": { "type": "string" },
        "AddressLine2": { "type": "string" },
        "City":         { "type": "string" }
    },
    "required": [ "AddressLine1" ],
    "oneOf": [
        {
          "type": "object",
          "additionalProperties": false,
          "properties": {
                "State":   { "type": "string" },
                "ZipCode": { "type": "string" }
            },
            "required": [ "ZipCode" ]
        },
        {
            "type": "object",
            "additionalProperties": false,
            "properties": {
                "County":   { "type": "string" },
                "PostCode": { "type": "string" }
            },
            "required": [ "PostCode" ]
        }
    ]
  }
}

The above schema generates the following models:

from __future__ import annotations

from typing import List, Optional, Union

from pydantic import BaseModel, Extra, Field


class LocationItem(BaseModel):
    class Config:
        extra = Extra.forbid

    State: Optional[str] = None
    ZipCode: str


class LocationItem1(BaseModel):
    class Config:
        extra = Extra.forbid

    County: Optional[str] = None
    PostCode: str


class Location(BaseModel):
    __root__: List[Union[LocationItem, LocationItem1]] = Field(..., title='My schema')

In which the AddressLine1, AddressLine2 and City are lost somewhere in the process. [problem2].

Expected behavior

  1. oneOf should make combinations of possible valid input data so that valid json payloads are parsable
  2. oneOf shouldn't drop properties when in a array. The generated classes should be very similar to oneOf without a list.

For problem1, the generated class should be:

from __future__ import annotations

from typing import Optional, Union

from pydantic import BaseModel, Extra, Field


class Boat(BaseModel):
    class Config:
        extra = Extra.forbid

    AddressLine1: str
    AddressLine2: Optional[str] = None
    City: Optional[str] = None
    State: Optional[str] = None
    ZipCode: str


class Boat1(BaseModel):
    class Config:
        extra = Extra.forbid

    AddressLine1: str
    AddressLine2: Optional[str] = None
    City: Optional[str] = None
    County: Optional[str] = None
    PostCode: str


class Boats(BaseModel):
    class Config:
        extra = Extra.forbid

    __root__: Union[Boat, Boat1] = Field(..., title='My schema')

In which the oneOf is a combination of the top fields, which validates and parses the the json payload.

and if problem1 is solved, then for problem2 the only change should be the __root__ to be:

__root__: List[Union[Boat, Boat1]] = Field(..., title="My schema")

Version:

  • OS: Linux Fedora 3.6
  • Python version: Python 3.10.6
  • datamodel-code-generator version: 0.13.1

Additional context Possible related comment: https://github.com/koxudaxi/datamodel-code-generator/issues/495#issuecomment-1109353969 Possible related issue: https://github.com/koxudaxi/datamodel-code-generator/issues/830

luminoso avatar Sep 16 '22 12:09 luminoso