datamodel-code-generator icon indicating copy to clipboard operation
datamodel-code-generator copied to clipboard

Published 20 hours ago • verified publisher icon koxudaxi

"A Parser can not resolve classes" error when generating code from an OpenAPI spec

Open jstasiak opened this issue 1 year ago • 4 comments

Hey all, first of all thank you all for coming up with this project and the continued work on it.

I bumped into the following crash this morning. I may attempt to investigate what's wrong but I figured it wouldn't hurt to report it right away.

Describe the bug datamodel-codegen produced an error instead of generating code:

% wget https://raw.githubusercontent.com/lune-climate/lune-docs/68726ac5b6e3b5c76cb683503311f0bb39d302f4/static/openapi.yml
--2024-05-22 14:33:42--  https://raw.githubusercontent.com/lune-climate/lune-docs/68726ac5b6e3b5c76cb683503311f0bb39d302f4/static/openapi.yml
Resolving raw.githubusercontent.com (raw.githubusercontent.com)... 185.199.110.133, 185.199.111.133, 185.199.108.133, ...
Connecting to raw.githubusercontent.com (raw.githubusercontent.com)|185.199.110.133|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 292341 (285K) [text/plain]
Saving to: ‘openapi.yml’

openapi.yml                                          100%[======================================================================================================================>] 285.49K  --.-KB/s    in 0.1s    

2024-05-22 14:33:42 (2.20 MB/s) - ‘openapi.yml’ saved [292341/292341]

% datamodel-codegen --input openapi.yml --input-file-type openapi --output models.py
Traceback (most recent call last):
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/__main__.py", line 447, in main
    generate(
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/__init__.py", line 468, in generate
    results = parser.parse()
              ^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 1187, in parse
    _, sorted_data_models, require_update_action_models = sort_data_models(
                                                          ^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 141, in sort_data_models
    return sort_data_models(
           ^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 141, in sort_data_models
    return sort_data_models(
           ^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 141, in sort_data_models
    return sort_data_models(
           ^^^^^^^^^^^^^^^^^
  [Previous line repeated 1 more time]
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 208, in sort_data_models
    raise Exception(f'A Parser can not resolve classes: {unresolved_classes}.')
Exception: A Parser can not resolve classes: [class: openapi.yml#/components/schemas/OrderQuoteByQuantityRequest references: frozenset({'openapi.yml#/components/schemas/OrderQuoteByQuantityWithBundleMass', 'openapi.yml#/components/schemas/OrderQuoteByQuantityWithBundlePercentage'})], [class: openapi.yml#/components/schemas/OrderQuoteByQuantityWithBundlePercentage references: frozenset({'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc', 'openapi.yml#/components/schemas/Mass'})], [class: openapi.yml#/components/schemas/OrderQuoteByQuantityWithBundleMass references: frozenset({'openapi.yml#/components/schemas/BundleMass', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/CreateOrderByQuantityRequest references: frozenset({'openapi.yml#/components/schemas/CreateOrderByQuantityWithBundleMass', 'openapi.yml#/components/schemas/CreateOrderByQuantityWithBundlePercentage'})], [class: openapi.yml#/components/schemas/CreateOrderByQuantityWithBundlePercentage references: frozenset({'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc', 'openapi.yml#/components/schemas/Mass'})], [class: openapi.yml#/components/schemas/CreateOrderByQuantityWithBundleMass references: frozenset({'openapi.yml#/components/schemas/BundleMass', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/OrderQuoteByValueRequest references: frozenset({'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/CreateOrderByValueRequest references: frozenset({'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc', 'openapi.yml#/components/schemas/Metadata'})], [class: openapi.yml#/components/schemas/CreateOrderByEstimateRequest references: frozenset({'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc', 'openapi.yml#/components/schemas/Metadata'})], [class: openapi.yml#/components/schemas/ElectricityEstimateRequest references: frozenset({'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/ElectricityConsumption', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/FlightEstimateRequest references: frozenset({'openapi.yml#/components/schemas/Distance', 'openapi.yml#/components/schemas/CabinClass', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/AirportSourceDestination', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/PassengerTransportationEstimateRequest references: frozenset({'openapi.yml#/components/schemas/PassengerFlightEstimateRequest', 'openapi.yml#/components/schemas/PassengerRoadEstimateRequest', 'openapi.yml#/components/schemas/PassengerRailEstimateRequest', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/PassengerTransportationEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/EstimateQuote', 'openapi.yml#/components/schemas/PassengerTransportationEstimateRequest', 'openapi.yml#/components/schemas/EmissionEstimate'})], [class: openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-allOf-#-special-# references: frozenset({'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/QuantityTrunc', 'openapi.yml#/components/schemas/ShippedAt', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/Shipment'})], [class: openapi.yml#/components/schemas/ShippingEstimateRequest references: frozenset({'openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-union_model-1-#-special-#', 'openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-union_model-0-#-special-#'})], [class: openapi.yml#/components/schemas/MultiLegShippingEstimateRequest references: frozenset({'openapi.yml#/components/schemas/ShippedAt', 'openapi.yml#/components/schemas/Shipment', 'openapi.yml#/components/schemas/MultiLegShippingEstimateRequest/legs#-datamodel-code-generator-#-array-#-special-#/0#-datamodel-code-generator-#-oneOf-#-special-#/0#-datamodel-code-generator-#-object-#-special-#', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/MultiLegShippingEstimateRequest/legs#-datamodel-code-generator-#-array-#-special-#/0#-datamodel-code-generator-#-oneOf-#-special-#/1#-datamodel-code-generator-#-object-#-special-#', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/BatchTransactionEstimateRequest references: frozenset({'openapi.yml#/components/schemas/TransactionEstimateRequest'})], [class: openapi.yml#/components/schemas/TransactionEstimateRequest references: frozenset({'openapi.yml#/components/schemas/Merchant', 'openapi.yml#/components/schemas/Diet', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/MonetaryAmount', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/CompanyEstimateRequest references: frozenset({'openapi.yml#/components/schemas/IntegerPercentage', 'openapi.yml#/components/schemas/CompanyEstimateRequest/tech#-datamodel-code-generator-#-object-#-special-#', 'openapi.yml#/components/schemas/Distance', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/MonetaryAmount', 'openapi.yml#/components/schemas/QuantityTrunc', 'openapi.yml#/components/schemas/Area'})], [class: openapi.yml#/components/schemas/ElectricityEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/EmissionEstimateResponse', 'openapi.yml#/components/schemas/ElectricityEstimateRequest'})], [class: openapi.yml#/components/schemas/FlightEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/EmissionEstimateResponse', 'openapi.yml#/components/schemas/FlightEstimateRequest'})], [class: openapi.yml#/components/schemas/BatchTransactionEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/EmptyObject', 'openapi.yml#/components/schemas/TransactionEmissionEstimate', 'openapi.yml#/components/schemas/ErrorResponse'})], [class: openapi.yml#/components/schemas/TransactionEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/EmissionEstimateResponse', 'openapi.yml#/components/schemas/EmissionFactorWithGasEmissions', 'openapi.yml#/components/schemas/NullEnum', 'openapi.yml#/components/schemas/TransactionEstimateRequest'})], [class: openapi.yml#/components/schemas/SingleShippingEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/ShippingEstimateRequest', 'openapi.yml#/components/schemas/EstimateQuote', 'openapi.yml#/components/schemas/ShippedAt', 'openapi.yml#/components/schemas/ShippingLegEmissionEstimate'})], [class: openapi.yml#/components/schemas/MultiLegShippingEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/Distance', 'openapi.yml#/components/schemas/AdjustedDistance', 'openapi.yml#/components/schemas/EmissionEstimateResponse', 'openapi.yml#/components/schemas/MultiLegShippingEstimateRequest', 'openapi.yml#/components/schemas/ShippedAt', 'openapi.yml#/components/schemas/ShippingLegEmissionEstimate'})], [class: openapi.yml#/components/schemas/AnyShippingEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/MultiLegShippingEmissionEstimate', 'openapi.yml#/components/schemas/SingleShippingEmissionEstimate'})], [class: openapi.yml#/components/schemas/PaginatedAnyShippingEmissionEstimates references: frozenset({'openapi.yml#/components/schemas/PaginatedBase', 'openapi.yml#/components/schemas/AnyShippingEmissionEstimate'})], [class: openapi.yml#/components/schemas/CompanyEmissionEstimate references: frozenset({'openapi.yml#/components/schemas/EmissionEstimateResponse', 'openapi.yml#/components/schemas/Mass', 'openapi.yml#/components/schemas/CompanyEstimateRequest', 'openapi.yml#/components/schemas/CompanyEmissionEstimate/components#-datamodel-code-generator-#-object-#-special-#'})], [class: openapi.yml#/components/schemas/EmissionFactorEstimateRequest references: frozenset({'openapi.yml#/components/schemas/EmissionFactorActivity', 'openapi.yml#/components/schemas/EstimateIdempotencyKey', 'openapi.yml#/components/schemas/Metadata', 'openapi.yml#/components/schemas/BundleSelectionRequest', 'openapi.yml#/components/schemas/QuantityTrunc'})], [class: openapi.yml#/components/schemas/EmissionFactorEstimate references: frozenset({'openapi.yml#/components/schemas/EmissionEstimateResponse', 'openapi.yml#/components/schemas/EmissionFactorEstimateRequest', 'openapi.yml#/components/schemas/EmissionFactorWithGasEmissions'})], [class: openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-union_model-0-#-special-# references: frozenset({'openapi.yml#/components/schemas/ShippingEstimateRequest/0#-datamodel-code-generator-#-object-#-special-#', 'openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-allOf-#-special-#'})], [class: openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-union_model-1-#-special-# references: frozenset({'openapi.yml#/components/schemas/ShippingEstimateRequest#-datamodel-code-generator-#-allOf-#-special-#', 'openapi.yml#/components/schemas/ShippingEstimateRequest/1#-datamodel-code-generator-#-object-#-special-#'})].

To Reproduce

The OpenAPI schema used (too large to include): https://raw.githubusercontent.com/lune-climate/lune-docs/68726ac5b6e3b5c76cb683503311f0bb39d302f4/static/openapi.yml

There may be something non-standard or broken about this specification that I'm not aware of but it is accepted by multiple other libraries/tools and it's been working fine so far.

Used commandline:

$ datamodel-codegen --input openapi.yml --input-file-type openapi --output models.py

Expected behavior I'd expect one of the following:

  1. An error message explaining some OpenAPI feature is used that is currently unsupported
  2. Code successfully generated

Version:

  • OS: macOS 13.4
  • Python version: 3.12.3
  • datamodel-code-generator version: 0.25.6
% pip freeze      
annotated-types==0.7.0
argcomplete==3.3.0
black==24.4.2
click==8.1.7
coverage==7.5.1
datamodel-code-generator==0.25.6
dnspython==2.6.1
email_validator==2.1.1
genson==1.3.0
idna==3.7
inflect==5.6.2
iniconfig==2.0.0
isort==5.13.2
Jinja2==3.1.4
MarkupSafe==2.1.5
mypy==1.10.0
mypy-extensions==1.0.0
packaging==24.0
pathspec==0.12.1
platformdirs==4.2.2
pluggy==1.5.0
pydantic==2.7.1
pydantic_core==2.18.2
pytest==8.2.1
pytest-cov==5.0.0
PyYAML==6.0.1
ruff==0.4.4
typing_extensions==4.11.0

Additional context N/A

jstasiak avatar May 22 '24 12:05 jstasiak

I managed to narrow it down to an input like this (openapi.yml):

openapi: 3.0.1

components:
  schemas:
    CreateOrderByEstimateRequest:
      type: object
      properties:
        quantity_trunc:
          $ref: '#/components/schemas/QuantityTrunc'

    QuantityTrunc:
      type: string
      description: Selects to which precision to truncate quantities specific to carbon offsetting.
      example: 't'
      allOf:
        - $ref: '#/components/schemas/MassUnit'

    MassUnit:
      type: string
      enum:
        - g
        - kg
        - t

The crash log:

% poetry run datamodel-codegen --input openapi.yml --input-file-type openapi --output models.py
Traceback (most recent call last):
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/__main__.py", line 447, in main
    generate(
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/__init__.py", line 468, in generate
    results = parser.parse()
              ^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 1187, in parse
    _, sorted_data_models, require_update_action_models = sort_data_models(
                                                          ^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 141, in sort_data_models
    return sort_data_models(
           ^^^^^^^^^^^^^^^^^
  File "/Users/user/Library/Caches/pypoetry/virtualenvs/spreadsheet-offset-tool-L93AmRO5-py3.12/lib/python3.12/site-packages/datamodel_code_generator/parser/base.py", line 208, in sort_data_models
    raise Exception(f'A Parser can not resolve classes: {unresolved_classes}.')
Exception: A Parser can not resolve classes: [class: openapi.yml#/components/schemas/CreateOrderByEstimateRequest references: frozenset({'openapi.yml#/components/schemas/QuantityTrunc'})].

I can't reduce the example any further.

  • If I remove the enum block from MassUnit the crash goes away
  • If I remove the CreateOrderByEstimateRequest type the crash goes away
  • If I remove QuantityTrunc and make CreateOrderByEstimateRequest refer to MassUnit (instead of QuantityTrunc) the crash goes away

So it seems like a $ref/anyOf/enum edge case?

jstasiak avatar May 23 '24 11:05 jstasiak

Interesting, if I change that allOf instance to oneOf (possible in this case, only one child type anyway), like

openapi: 3.0.1

components:
  schemas:
    CreateOrderByEstimateRequest:
      type: object
      properties:
        quantity_trunc:
          $ref: '#/components/schemas/QuantityTrunc'

    QuantityTrunc:
      type: string
      description: Selects to which precision to truncate quantities specific to carbon offsetting.
      example: 't'
      oneOf:
        - $ref: '#/components/schemas/MassUnit'

    MassUnit:
      type: string
      enum:
        - g
        - kg
        - t

the tool actually generates code:

# generated by datamodel-codegen:
#   filename:  openapi.yml
#   timestamp: 2024-05-23T11:41:39+00:00

from __future__ import annotations

from enum import Enum
from typing import Optional

from pydantic import BaseModel, Field


class MassUnit(Enum):
    g = 'g'
    kg = 'kg'
    t = 't'


class QuantityTrunc(BaseModel):
    __root__: MassUnit = Field(
        ...,
        description='Selects to which precision to truncate quantities specific to carbon offsetting.',
        example='t',
    )


class CreateOrderByEstimateRequest(BaseModel):
    quantity_trunc: Optional[QuantityTrunc] = None

jstasiak avatar May 23 '24 11:05 jstasiak

Hi. I'm also facing this issue. Did you find a way to resolve it other than replacing allOf with oneOf?

dpeachey avatar Jul 08 '24 11:07 dpeachey

Hey @dpeachey, no, I had to get things done quickly so because of this and some other issues I skipped code generation completely and just wrote the types I needed by hand. :|

jstasiak avatar Jul 08 '24 11:07 jstasiak