When RAML includes JSON schema which $ref separate sub-schema (0.8 okay, 1.0 fails)

[dcrossleyau]

The attached set of example files do "!include" a JSON schema file in another directory, and that schema refers to another schema in a separate file.

The top-level schema examples use two different methods to declare the second JSON schema:

1. By filename -- "$ref": "pet.json"

2. By schema key -- "$ref" : "petSchema" (i.e. as declared in the top-level RAML file)

Using raml-parser v0.8.17 -- both methods are successful for RAML-0.8 version.

Using raml-parser-2 v1.0.7 -- both methods are successful for RAML-0.8 version.

Using raml-parser-2 v1.0.7 -- both methods fail for RAML-1.0 version. Error validating JSON. Error: Invalid json content : ... (An obscure message, but it probably means that it cannot find the second schema file.)

Also tested back to v1.0.0 and same problem.

The parent schema files do validate okay using 'ajv', so they are self-contained.

pets.tar.gz

test-pets/
|
|__ examples/
|   |__ valid-pet.json
|   |__ valid-pets.json
|
|__ ramls/
|   |__ 08-4-included-schema-filename.raml
|   |__ 08-5-included-schema-flat.raml
|   |__ 08-6-included-schema-key-id.raml
|   |__ 10-4-included-schema-filename.raml
|   |__ 10-5-included-schema-flat.raml
|   |__ 10-6-included-schema-key-id.raml
|
|__ schema/
    |__ pet.json
    |__ pets-filename.json (includes "pet" via "$ref": "pet.json")
    |__ pets-flat.json
    |__ pet-key-id.json
    |__ pets-key-id.json (includes "pet" via "$ref" : "petSchema")

Aha! Link: https://mulesoft-roadmap.aha.io/features/APIRAML-96

[dcrossleyau]

A related issue seems to be #310

That issue has PR 311, whose branch also has the same behaviour.

[dcrossleyau]

Also tried with adding a "hash" symbol to the schema $ref ...

diff schema/pets-filename.json schema/pets-filename-id-hash.json
3c3
<   "id": "pets-filename.json",
---
>   "id": "pets-filename-id-hash.json#",
11c11
<         "$ref" : "pet.json"
---
>         "$ref": "pet-id-hash.json#"

The behaviour is still the same as described above.

[dcrossleyau]

Following the recent merge of #311, the behaviour is still the same as described above.

[myhau]

So, after looong debugging session I think I know the root cause and a solution.

Problem:

In SchemaGenerator.generateJsonSchema

this fragment of code is what would make json schema $ref's work

String includedResourceUri = resolveResourceUriIfIncluded(jsonTypeDefinition);
...
if (includedResourceUri != null)
            {
                return factory.getJsonSchema(includedResourceUri);
            }
...

and in resolveResourceUriIfIncluded you can see:

return schema.getStartPosition().getIncludedResourceUri();

the problem is schema.getStartPosition().getIncludedResourceUri() is null, because

ExternalSchemaTypeExpressionNode.setSource (which sets start position) is never called

and according to Node docs:

/**
     * The source of this node. This is used at transformation phases when the original Yaml node are being replaced by a more specialized node.
     * This way the node specialization can be tracked on all the history changes.
     * @param source The original node.
     */
    void setSource(Node source);

So it should have been called when transforming StringNode to ExternalSchemaTypeExpressionNode

in SchemaDeclarationRule

return createNodeUsingFactory(node, ((StringNode) node).getValue());

A lot of mental shortcuts there, please let me know if it is understandable at all

My solution (ugly, I don't know this codebase):

In SchemaDeclarationRule, change:

    @Nonnull
    @Override
    public Node apply(@Nonnull Node node)
    {
        if (matches(node))
        {
            return createNodeUsingFactory(node, ((StringNode) node).getValue());
        }
        else
        {
            return ErrorNodeFactory.createInvalidNode(node);
        }
    }

to

    @Nonnull
    @Override
    public Node apply(@Nonnull Node node)
    {
        if (matches(node))
        {
            Node newNode = createNodeUsingFactory(node, ((StringNode) node).getValue());

            if(newNode != node) {
                newNode.setSource(node);
            }

            return newNode;
        }
        else
        {
            return ErrorNodeFactory.createInvalidNode(node);
        }
    }

The problem is that 5 tests fail:

If anyone wants to implement it and knows exactly why existing test fail, I can provide more details, just ask or contact me. I can also create a PR, but I need to understand how to fix related tests, @machaval can you help ?