jsonschema

jsonschema copied to clipboard

When using a custom scheme, such as custom://, intra-schema relative references will fail to resolve. They only fail, however, in schemas referenced from the base schema passed to the validator.

Minimal Repro Example

foo/base.json

{
    "$ref": "custom://bar.json"
}

foo/bar.json

{
    "definitions": {
        "bar": {
            "type": "number"
        }
    },
    "$ref": "#/definitions/bar"
}

repro.py

import jsonschema
import json
import os

scheme = 'custom'


def handler(url):
    prefix = '{}://'.format(scheme)
    assert(url.startswith(prefix))
    url = url[len(prefix):]

    with open(os.path.join('foo', url), 'r') as f:
        return json.load(f)


schema = handler('custom://base.json')
resolver = jsonschema.RefResolver.from_schema(schema, handlers={scheme: handler})
errors = jsonschema.Draft7Validator(schema, resolver=resolver).iter_errors(0)
print(list(errors))

Repro Stacktrace

Traceback (most recent call last):
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 811, in resolve_fragment
    document = document[part]
KeyError: 'definitions'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/private/tmp/repro/repro.py", line 20, in <module>
    print(list(errors))
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/_validators.py", line 263, in ref
    for error in validator.descend(instance, resolved):
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 344, in descend
    for error in self.iter_errors(instance, schema):
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 328, in iter_errors
    for error in errors:
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/_validators.py", line 259, in ref
    scope, resolved = validator.resolver.resolve(ref)
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 766, in resolve
    return url, self._remote_cache(url)
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 781, in resolve_from_url
    return self.resolve_fragment(document, fragment)
  File "/private/tmp/repro/lib/python3.7/site-packages/jsonschema/validators.py", line 814, in resolve_fragment
    "Unresolvable JSON pointer: %r" % fragment
jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'definitions/bar

Cause

This is caused by a failure of urllib to allow relative paths in urljoin when scheme is not in uses_relatives. As a result, the call to self._urljoin_cache simply returns ref and the reference fails to resolve.

Stacktrace of cause of issue:

jsonschema/validators.py:328 iter_errors
jsonschema/_validators.py:259 ref
jsonschema/validators.py:765 resolve
...
urllib/parse.py:493 urljoin

The cause of this bug was discussed in #104, specifically by @gazpachoking, here as:

Possibly also do this for urljoin and add the scheme to urlparse.uses_relative and urlparse.uses_netloc too. This would apply to python 3.3 as well.

However, this note was not addressed in #106, which fixed #104.

System Info

• OS: MacOS 10.14.1
• Python: 3.7.6
• jsonschema: 3.2.0

Does this happen even with requests installed?

Yes, just confirmed. Repro steps:

mkdir foo
echo "..." > foo/base.json
echo "..." > foo/bar.json
echo "..." > repro.py
python3 -m venv .
source bin/activate
pip3 install requests jsonschema

python3 repro.py

This looks fixed, probably by d9e093d439ab3023ffb644d620155e277032740c or the earlier fix for URN-related IDs.

Feel free to follow up if not but the reproducer here seems to properly spit out [] now.

Thanks for filing.