Failing to parse open-rpc's meta-schema

[vesajaaskelainen]

trafficstars

Describe the bug open-rpc provides meta-schema in following locations: https://github.com/open-rpc/meta-schema/ and https://meta.open-rpc.org/

Now in order to parse files based on this meta-schema one would like to generate data model for it.

There seems to be multiple errors in handling open-rpc meta-schema and referenced json-schema-tools's meta-schema:

• reference is not working with absolute https uri.
• json-schema-tools's schema.json 'items' keyword in 'enum' and in 'examples' array which does not work
• json-schema-tools's schema.json 'const' keyword does not work
• json-schema-tools's schema.json 'default' keyword in JSONSchemaObject does not work

To Reproduce

Example schema:

# use directly from web: 
# https://meta.open-rpc.org/
# or
# download locally:
wget https://meta.open-rpc.org/ -O open-rpc-meta-schema.json
# or fetch latest release from github:
wget https://github.com/open-rpc/meta-schema/releases/download/1.14.2/open-rpc-meta-schema.json

Used commandline:

(venv) user@host:/tmp/test$ datamodel-codegen --url https://meta.open-rpc.org/ --input-file-type jsonschema --output openrpc.py
Traceback (most recent call last):
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/__main__.py", line 495, in main
    generate(
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/__init__.py", line 353, in generate
    results = parser.parse()
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/base.py", line 426, in parse
    self.parse_raw()
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1177, in parse_raw
    self._parse_file(self.raw_obj, obj_name, path_parts)
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1255, in _parse_file
    self.parse_raw_obj(key, model, path)
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1122, in parse_raw_obj
    self.parse_obj(name, JsonSchemaObject.parse_obj(raw), path)
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1142, in parse_obj
    self.parse_ref(obj, path)
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1064, in parse_ref
    self.resolve_ref(obj.ref)
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1053, in resolve_ref
    self._parse_file(
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1235, in _parse_file
    with self.root_id_context(raw):
  File "/usr/lib/python3.8/contextlib.py", line 113, in __enter__
    return next(self.gen)
  File "/tmp/test/venv/lib/python3.8/site-packages/datamodel_code_generator/parser/jsonschema.py", line 1110, in root_id_context
    root_id: Optional[str] = root_raw.get('$id')
AttributeError: 'str' object has no attribute 'get'

Expected behavior It generates usable openrpc.py file.

Version:

• OS: ubuntu 20.04 LTS
• Python version: 3.8.10
• datamodel-code-generator version: 0.11.14

Additional context N/A

[vesajaaskelainen]

With added debug for reference problem:

diff --git a/datamodel_code_generator/parser/jsonschema.py b/datamodel_code_generator/parser/jsonschema.py
index fe0e320..bb0e006 100644
--- a/datamodel_code_generator/parser/jsonschema.py
+++ b/datamodel_code_generator/parser/jsonschema.py
@@ -44,6 +44,7 @@ from datamodel_code_generator.parser.base import (
 )
 from datamodel_code_generator.reference import ModelType, Reference, is_url
 from datamodel_code_generator.types import DataType, DataTypeManager, StrictTypes, Types
+import json
 
 
 def get_model_by_path(schema: Dict[str, Any], keys: List[str]) -> Dict[str, Any]:
@@ -196,6 +197,7 @@ class JsonSchemaObject(BaseModel):
         keep_untouched = (cached_property,)
 
     def __init__(self, **data: Any) -> None:  # type: ignore
+        print("%s" % json.dumps(data))
         super().__init__(**data)
         self.extras = {k: v for k, v in data.items() if k not in EXCLUDE_FIELD_KEYS}
 

We can see that following is getting parsed:

{"$ref": "https://raw.githubusercontent.com/json-schema-tools/meta-schema/1.5.9/src/schema.json"}

And results in reference as "<original source>/https://..." which is incorrect and there should only be absolute URL in there.

[vesajaaskelainen]

For reference problem there is a PR opened.

[vesajaaskelainen]

With the reference fix in we do get new set of errors:

pydantic.error_wrappers.ValidationError: 6 validation errors for JsonSchemaObject
properties -> default
  value is not a valid dict (type=type_error.dict)
properties -> examples -> items
  value is not a valid list (type=type_error.list)
properties -> examples -> items
  value is not a valid dict (type=type_error.dict)
properties -> const
  value is not a valid dict (type=type_error.dict)
properties -> enum -> items
  value is not a valid list (type=type_error.list)
properties -> enum -> items
  value is not a valid dict (type=type_error.dict)

If one goes to modify json-schema-tools a bit:

--- ../schema.json	2021-10-31 09:49:14.978432793 +0200
+++ schema.json	2021-10-31 09:57:19.000000000 +0200
@@ -44,7 +44,6 @@
           "title": "description",
           "type": "string"
         },
-        "default": true,
         "readOnly": {
           "title": "readOnly",
           "type": "boolean",
@@ -52,8 +51,7 @@
         },
         "examples": {
           "title": "examples",
-          "type": "array",
-          "items": true
+          "type": "array"
         },
         "multipleOf": {
           "title": "multipleOf",
@@ -168,11 +166,9 @@
         "propertyNames": {
           "$ref": "#"
         },
-        "const": true,
         "enum": {
           "title": "enum",
           "type": "array",
-          "items": true,
           "minItems": 1,
           "uniqueItems": true
         },

And then trying it out:

$ datamodel-codegen --input open-rpc-meta-schema.json --input-file-type jsonschema --output openrpc.py

It now generates openrpc.py file but it seems to have challenges with openrpc examples:

• https://raw.githubusercontent.com/open-rpc/examples/master/service-descriptions/empty-openrpc.json -> fine
• https://raw.githubusercontent.com/open-rpc/examples/master/service-descriptions/petstore-openrpc.json -> validation errors
• https://raw.githubusercontent.com/open-rpc/examples/master/service-descriptions/simple-math-openrpc.json -> validation errors
• Also fails with others

I am able to craft a custom file that gets parsed but then some details are not found like properties for result and sometimes also for parameters.

I suppose there is some generic stuff broken in a way that openrpc files are being specified almost looks like another reference problem but this time inside the file? Perhaps those official open-rpc example files gives a clue what could be wrong.

[koxudaxi]

@vesajaaskelainen I'm sorry for my too-late reply.

- "items": true,

Does JsonSchema support boolean for items?

[koxudaxi]

@vesajaaskelainen
I'm sorry for my too late reply I have released the PR as 0.15.0 Thank you very much!!