Problems (and solutions) for generating a working model for AWS CloudFormation

[keean]

Describe the bug datamodel-codegen is unable to produce a working dataclass model for AWS CloudFormation.

• ordering of output class properties does not work with default values and inheritance, suggest adding kw_only=True to @dataclass decorator.
• mutable default values are not allowed, so default Lists and Dicts need to be wrapped in: f'field(default_factory=lambda: {default} )'
• the type aliases do not get generated in the correct positions, for example Conditions needs to be declared after ConditionalExpression which in turn needs to be declared after all the union members are defined. Below you can see the output as it is now:

Conditions = Dict[str, ConditionalExpression]

ConditionalExpression = Union[str, FnAnd, FnEquals, FnNot, FnOr, FnFindInMap, FnRef]

@dataclass(kw_only=True)
class FnFindInMap:
    Fn__FindInMap: Optional[List[Union[str, Expression]]] = None

• needs to import dataclass from pydantic.dataclasses for model checking, validation and serialisation using pydantic. This is necessary for any generated dataclass model to work with fastapi.
• There is an odd error TypeError: 'NoneType' object is not subscriptable that seems to be caused by having an object property name the same as the type name... This fails with the above error:

@dataclass(kw_only=True)
class EksSecret:
    SecretName: str
    Optional: Optional[bool] = None

This works:

OptProp = Optional

@dataclass(kw_only=True)
class EksSecret:
    SecretName: str
    Optional: OptProp[bool] = None

As soon as there is a property called 'optional' we have to replace all uses of Optional types in that class with the alias for it to work. Maybe this is a pydantic bug that needs to be reported upstream, but this work around gets things working.

• There is some issue around type aliases:

DeletionProtected = bool

@dataclass(kw_only=True)
class AwsSsmincidentsReplicationset:
    Regions: RegionList
    DeletionProtected: Optional[DeletionProtected] = False
    Tags: Optional[List[TagModel19]] = field(default_factory=lambda: [])

This fails with: pydantic.errors.PydanticSchemaGenerationError: Unable to generate pydantic-core schema for False. Set arbitrary_types_allowed=True in the model_config to ignore this error or implement __get_pydantic_core_schema__ on your type to fully support it.

This appears to be related to the property name / type name issue, as the following works:

DelProt = bool

@dataclass(kw_only=True)
class AwsSsmincidentsReplicationset:
    Regions: RegionList
    DeletionProtected: Optional[DelProt] = False
    Tags: Optional[List[TagModel19]] = field(default_factory=lambda: [])

To Reproduce copy the schema files locally (using --url does not seem to follow the references properly resulting in a few hundred lines in the model, whereas it should be nearly 60,000 lines

If all the changed above are applied by hand to the generated model, it then works with fastapi.

Example schema: https://github.com/aws-cloudformation/cfn-lint-visual-studio-code/blob/main/server/schema/base.schema.json

Used commandline:

datamodel-codegen --output-model-type dataclasses.dataclass --enum-field-as-literal all --use-one-literal-as-default --input schema/base.schema.json --output base_schema.py

Expected behavior model generated to work with fastapi Version:

• OS: Linux
• Python version: 3.11.5
• datamodel-code-generator version: [e.g. 22] 0.22.0

Additional context Started to discuss some of these issues in a different issue, but there is little value to me in just fixing one of these issues, the value is in being able to use the generated model with fastapi, which needs all of these issues to be fixed together.

I am happy to provide additional context, do further debugging, run tests, help resolve this in any way I can.