python-openapi
openapi_core.create_spec fails to resolve $ref to parent directory
I have a schema with a reference graph that looks like openapi.yaml -> sub/paths.yaml -> schemas.yaml. The middle schema is in a subdirectory and the first and last schemas are in the same directory. This fails to resolve correctly when proving the spec_dict generated from openapi.yaml to create_spec in the following way:
from pathlib import Path
from openapi_core import create_spec
from openapi_spec_validator.schemas import read_yaml_file
path = Path("relative-reference/openapi.yaml").resolve()
create_spec(read_yaml_file(path), spec_url=path.as_uri())
The three OpenAPI files are as follows:
relative-reference/openapi.yaml:
openapi: "3.0.0"
info:
version: "1.0.0"
title: Relative references not resolving correctly
paths:
/:
$ref: "./sub/paths.yaml#/root"
relative-reference/sub/paths.yaml:
root:
get:
summary: Some API endpoint
responses:
'200':
description: Content for the API
content:
application/json:
schema:
type: array
items:
$ref: ../schemas.yaml#/Item
relative-reference/schemas.yaml:
Item:
type: object
required: ["name"]
properties:
name:
type: string
maxLength: 40
The error I get ends with
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 754, in resolving
url, resolved = self.resolve(ref)
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 766, in resolve
return url, self._remote_cache(url)
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 779, in resolve_from_url
raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: <urlopen error [Errno 2] No such file or directory: '/workdir/schemas.yaml'>
The path it tries to resolve lacks the relative-reference directory name.
However, fixing it by adding the missing path portion in the reference does not help. When changing the parent-directory reference to ../relative-reference/schemas.yaml#/Item, the error changes but remains:
jsonschema.exceptions.RefResolutionError: <urlopen error [Errno 2] No such file or directory: '/workdir/relative-reference/relative-reference/schemas.yaml'>
The 'base' directory was left on the path and now we have a failure because we try to access a directory that doesn't exist. Adding a differently named path portion here results in the same problem (except the failed path will be e.g. /workdir/relative-reference/dummy/schemas.yaml. This error happens on all Pythons 3.6 through 3.9.
The full track trace:
Traceback (most recent call last):
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 774, in resolve_from_url
document = self.store[url]
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/_utils.py", line 22, in __getitem__
return self.store[self.normalize(uri)]
KeyError: 'file:///workdir/schemas.yaml'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.7/urllib/request.py", line 1506, in open_local_file
stats = os.stat(localfile)
FileNotFoundError: [Errno 2] No such file or directory: '/workdir/schemas.yaml'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 777, in resolve_from_url
document = self.resolve_remote(url)
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 856, in resolve_remote
result = self.handlers[scheme](uri)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_spec_validator/handlers.py", line 35, in __call__
f = urlopen(url, timeout=timeout)
File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.7/urllib/request.py", line 525, in open
response = self._open(req, data)
File "/usr/local/lib/python3.7/urllib/request.py", line 543, in _open
'_open', req)
File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.7/urllib/request.py", line 1484, in file_open
return self.open_local_file(req)
File "/usr/local/lib/python3.7/urllib/request.py", line 1523, in open_local_file
raise URLError(exp)
urllib.error.URLError: <urlopen error [Errno 2] No such file or directory: '/workdir/schemas.yaml'>
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "example.py", line 7, in <module>
create_spec(read_yaml_file(path), spec_url=path.as_uri())
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/shortcuts.py", line 20, in create_spec
return spec_factory.create(spec_dict, spec_url=spec_url)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/specs/factories.py", line 49, in create
info, list(paths), servers=list(servers), components=components,
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/paths/generators.py", line 38, in generate
path_name, list(operations), parameters=list(parameters),
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/operations/generators.py", line 77, in generate
servers=list(servers), extensions=extensions,
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/operations/models.py", line 16, in __init__
self.responses = dict(responses)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/responses/generators.py", line 40, in generate
extensions=extensions,
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/responses/models.py", line 15, in __init__
self.content = content and Content(content) or Content()
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/media_types/generators.py", line 30, in generate
schema, _ = self.schemas_registry.get_or_create(schema_spec)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/schemas/registries.py", line 28, in get_or_create
schema = self.create(schema_deref)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/schemas/factories.py", line 68, in create
items = self._create_items(items_spec)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/schemas/factories.py", line 101, in _create_items
return self.create(items_spec)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_core/schema/schemas/factories.py", line 21, in create
schema_deref = self.dereferencer.dereference(schema_spec)
File "/workdir/.pip/lib/python3.7/site-packages/openapi_spec_validator/validators.py", line 36, in dereference
with resolver.resolving(ref) as target:
File "/usr/local/lib/python3.7/contextlib.py", line 112, in __enter__
return next(self.gen)
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 754, in resolving
url, resolved = self.resolve(ref)
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 766, in resolve
return url, self._remote_cache(url)
File "/workdir/.pip/lib/python3.7/site-packages/jsonschema/validators.py", line 779, in resolve_from_url
raise exceptions.RefResolutionError(exc)
jsonschema.exceptions.RefResolutionError: <urlopen error [Errno 2] No such file or directory: '/workdir/schemas.yaml'>
Relevant package versions:
jsonschema==3.2.0
openapi-core==0.13.7
openapi-schema-validator==0.1.2
openapi-spec-validator==0.2.9
Digging in to this a bit more, it seems as if the resolution_scope of the RefResolver provided by jsonschema has the wrong scope to resolve the URL correctly. That said, I don't quite comprehend the entire path leading up to this point, so I'm not sure whether this is painfully obvious, a red herring, or actually of some use:
resolution_scope and ref in jsonschema.RefResolver.resolve for various different schema references:
$ref: "../schemas.yaml#/Item"
resolution scope: file:///workdir/relative-reference/openapi.yaml
ref: ../schemas.yaml#/Item
$ref: "./schemas.yaml#/Item"
resolution scope: file:///workdir/relative-reference/sub/paths.yaml#/root
ref: ./schemas.yaml#/Item
$ref: "schemas.yaml#/Item"
resolution scope: file:///workdir/relative-reference/sub/paths.yaml#/root
ref: schemas.yaml#/Item