OpenAPI - Paths $ref Does Not Resolve Correctly

[zach-hamm]

Describe the bug There appears to be a bug when it comes to local file references; specifically for the paths section. While iterating over the paths object, there is a piece of code which resolves the references:

https://github.com/koxudaxi/datamodel-code-generator/blob/739b05004d2f88896824c31cbad3897502369769/datamodel_code_generator/parser/openapi.py#L572-L573

The reference resolution here works as expected, but it loses the context of where the reference is from. In other words, if I was pointing to a file at ../../my-file.yaml then that path may no longer be valid. For example, let's say that we have a simple example as follows:

To Reproduce (1st case)

Folder structure:

/
  openapi.yaml
  paths/
    test.yaml
  components/
    schemas/
      test_object.yaml

openapi.yaml:

openapi: 3.1.0
info:
  title: OpenAPI
  description: OpenAPI
  version: 1.0.0
paths:
  /test:
    $ref: ./paths/test.yaml
components:
  schemas:
    test_object:
      $ref: ./components/schemas/test_object.yaml
servers:
  - url: http://localhost:3000

paths/test.yaml:

get:
  responses:
    "200":
      description: "OK"
      content:
        application/json:
          schema:
            $ref: "../components/schemas/test_object.yaml"

components/schemas/test_object.yaml:

type: object
properties:
  name:
    type: string

Used commandline:

$ datamodel-codegen --input-file-type openapi --input openapi.yaml --output t.py --openapi-scopes schemas paths parameters

Expected behavior In a normal scenario, since I've specified the paths and parameters scopes for openapi I would expect that the library provides Pydantic models as a result. However, what ends up happening is nothing gets created at all. The library doesn't report an error, but I would expect something to be created for this path. Here's what I got:

# generated by datamodel-codegen:
#   filename:  openapi.yaml
#   timestamp: 2023-09-28T00:09:01+00:00

from __future__ import annotations

from typing import Optional

from pydantic import BaseModel


class TestObject(BaseModel):
    name: Optional[str] = None

To Reproduce (2nd case)

For the 2nd case, leave everything the exact same, but we're going to change how our path is setup. Rather than referencing a schema directly, we will instead create a new object that has references like so:

paths/test.yaml:

get:
  responses:
    "200":
      description: "OK"
      content:
        application/json:
          schema:
            type: object
            properties:
              data:
                $ref: "../components/schemas/test_object.yaml"

Used commandline:

$ datamodel-codegen --input-file-type openapi --input openapi.yaml --output t.py --openapi-scopes schemas paths parameters

Expected behavior What happens in this case is it attempts to find the ../components/schemas/test_object.yaml file, but it appends that route onto where the openapi.yaml file is which makes the first route invalid. It should have started at ./paths, but it started at ./.

Redacted my computers file path and swapped it with ...

Traceback (most recent call last):
  File ".../datamodel_code_generator/__main__.py", line 388, in main
    generate(
  File ".../datamodel_code_generator/__init__.py", line 435, in generate
    results = parser.parse()
  File ".../datamodel_code_generator/parser/base.py", line 1058, in parse
    self.parse_raw()
  File ".../datamodel_code_generator/parser/openapi.py", line 592, in parse_raw
    self.parse_operation(
  File ".../datamodel_code_generator/parser/openapi.py", line 522, in parse_operation
    self.parse_responses(
  File ".../datamodel_code_generator/parser/openapi.py", line 371, in parse_responses
    data_types[status_code][content_type] = self.parse_schema(
  File ".../datamodel_code_generator/parser/openapi.py", line 326, in parse_schema
    self.parse_ref(obj, path)
  File ".../datamodel_code_generator/parser/jsonschema.py", line 1498, in parse_ref
    self.parse_ref(property_value, path)
  File ".../datamodel_code_generator/parser/jsonschema.py", line 1476, in parse_ref
    self.resolve_ref(obj.ref)
  File ".../datamodel_code_generator/parser/jsonschema.py", line 1466, in resolve_ref
    self._get_ref_body(relative_path),
  File ".../datamodel_code_generator/parser/jsonschema.py", line 1418, in _get_ref_body
    return self._get_ref_body_from_remote(resolved_ref)
  File ".../datamodel_code_generator/parser/jsonschema.py", line 1431, in _get_ref_body_from_remote
    return self.remote_object_cache.get_or_put(
  File ".../datamodel_code_generator/parser/__init__.py", line 28, in get_or_put
    value = self[key] = default_factory(key)
  File ".../datamodel_code_generator/parser/jsonschema.py", line 1433, in <lambda>
    default_factory=lambda _: load_yaml_from_path(full_path, self.encoding),
  File ".../datamodel_code_generator/__init__.py", line 51, in load_yaml_from_path
    with path.open(encoding=encoding) as f:
  File "/opt/homebrew/Cellar/[email protected]/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1252, in open
    return io.open(self, mode, buffering, encoding, errors, newline,
  File "/opt/homebrew/Cellar/[email protected]/3.9.18/Frameworks/Python.framework/Versions/3.9/lib/python3.9/pathlib.py", line 1120, in _opener
    return self._accessor.open(self, flags, mode)
FileNotFoundError: [Errno 2] No such file or directory: './../components/schemas/test_object.yaml'

Version:

• OS: MacOS - Ventura 13.6
• Python version: 3.9
• datamodel-code-generator version: 0.22.0

Additional context Again, I believe the issue comes from the fact that for path resolution specifically, the library resolves the reference locally and "thinks" that everything that was resolved is therefore wherever openapi.yaml is. However, that's not always the case as a path could be referenced under a different directory. Since that context is lost, any references inside of that path file will then be invalid since the starting location has changed.

[koxudaxi]

Thank you for investigating the issue. OK, We should fix the bug.

[zjhamm]

Apologies, I've been out for quite some time, but I'm checking back in on this. Has there been any progress here? I just downloaded the latest version (0.25.4) and it looks like this issue still exists.