RefResolutionError on PyPy only

[jpmckinney]

To reproduce:

$ python --version
Python 3.7.10 (51efa818fd9b, Apr 04 2021, 12:03:51)
[PyPy 7.3.4 with GCC Apple LLVM 12.0.0 (clang-1200.0.32.29)]

pip install jsonschema requests

import jsonschema

instance = [[{}], [{}]]
schema = {
    "items": {"$ref": "#/definitions/record"},
    "definitions": {
        "record": {
            "oneOf": [
                {},
                {"$ref": "https://standard.open-contracting.org/schema/1__1__5/release-schema.json"}
            ]
        }
    }
}

Causes:

jsonschema.exceptions.RefResolutionError: Unresolvable JSON pointer: 'definitions/record'

The error does not occur on CPython.

The underlying cause is likely here, where we're improperly relying on the pop_scope call to run immediately on the suspended generator, whereas it really can run anytime afterwards (and does on PyPy), causing the current resolution scope to be incorrect when resolving the ref.

[dimrozakis]

I'm also getting the same error while using a RefResolver to handle referenced schemas that exist as local files. The code works properly on cpython but when run under pypy, it will try to resolve a second reference from within the previous reference's directory.

[Julian]

Hello there!

This, along with many many other $ref-related issues, is now finally being handled in #1049 with the introduction of a new referencing library which is fully compliant and has APIs which I hope are a lot easier to understand and customize.

On PyPy specifically, besides being even faster than before, the new APIs don't rely on bad CPython behavior like garbage collection of generators :/ -- yay!

The next release of jsonschema (v4.18.0) will contain a merged version of that PR, and should be released shortly in beta, and followed quickly by a regular release, assuming no critical issues are reported.

It looks from my testing like indeed this specific example works there! If you still care to, I'd love it if you tried out the beta once it is released, or certainly it'd be hugely helpful to immediately install the branch containing this work (https://github.com/python-jsonschema/jsonschema/tree/referencing) and confirm. You can in the interim find documentation for the change in a preview page here.

I'm going to close this given it indeed seems like it is addressed by #1049, but feel free to follow up with any comments. Sorry for the delay in getting to these, but hopefully this new release will bring lots of benefit!

(Here's the modification, in case it helps:

from referencing import Registry, Resource
import httpx
import jsonschema

instance = [[{}], [{}]]
schema = {
    "items": {"$ref": "#/definitions/record"},
    "definitions": {
        "record": {
            "oneOf": [
                {},
                {"$ref": "https://standard.open-contracting.org/schema/1__1__5/release-schema.json"}
            ]
        }
    }
}

response = httpx.get("https://standard.open-contracting.org/schema/1__1__5/release-schema.json")
resource = Resource.from_contents(response.json())
registry = Registry().with_resource(
    "https://standard.open-contracting.org/schema/1__1__5/release-schema.json",
    resource,
)
jsonschema.validate(schema=schema, instance=instance, registry=registry)

)