ajv
ajv copied to clipboard
ajv-validator
$recursiveRef: only supports hash fragment reference
AJV: 8.6.2 Draft: 2019-09
Hi, I've run into into a issue with $recursiveAnchor and $recursiveRef where I'm unable to target non-root schemas using $recursiveRef. For simplicity, below is the TypeScript data structure I'm trying to represent in JSON schema.
interface Element {
elementId: string
elements: Element[]
}
interface Node {
nodeId: string
nodes: Node[]
element: Element
}
The following is the JSON schema I'm using to represent the above type. I believe JSON schema interprets the # pointer in the following to refer to the root schema in nested schemas, causing the inner Element array to be interpreted as an array of type Node. This is expected behaviour.
{
"additionalProperties": false,
"$id": "Node", // <------------+
"$recursiveAnchor": true, // |
"type": "object", // |
"properties": { // |
"nodeId": { // |
"type": "string" // |
}, // |
"nodes": { // |
"type": "array", // | { $recursiveRef: '#' } targets root schema "Node"
"items": { // |
"$recursiveRef": "#" // -------------+
} // |
}, // |
"element": { // |
"additionalProperties": false, // |
"$id": "Element", // |
"$recursiveAnchor": true, // |
"type": "object", // |
"properties": { // |
"elementId": { // |
"type": "string" // |
}, // |
"elements": { // |
"type": "array", // |
"items": { // |
"$recursiveRef": "#" // -------------+
}
}
}
}
}
}
To address this, I've tried passing an explicit JSON pointer to try and target the appropriate sub schema.
{
"additionalProperties": false,
"$id": "Node", // <------------+
"$recursiveAnchor": true, // |
"type": "object", // | Array of Node
"properties": { // |
"nodeId": { // |
"type": "string" // |
}, // |
"nodes": { // |
"type": "array", // |
"items": { // |
"$recursiveRef": "Node#" // -------------+
}
},
"element": {
"additionalProperties": false,
"$id": "Element", // <------------+
"$recursiveAnchor": true, // |
"type": "object", // | Array of Element
"properties": { // |
"elementId": { // |
"type": "string" // |
}, // |
"elements": { // |
"type": "array", // |
"items": { // |
"$recursiveRef": "Element#" // -------------+
}
}
}
}
}
}
However, AJV is reporting the following for the second schema.
Error: "$recursiveRef" only supports hash fragment reference
Not sure if this is against specification, or a possible constraint with some of the architectural changes in AJV to support these newer drafts. I've come across https://github.com/ajv-validator/ajv/issues/1198 which suggests dynamic ref scoping (which I'm guessing relates to this) So not sure if this is working as intended or is a bug. However, I've tested the second schema in JSON.NET Schema which seems to support referencing of this kind. So not sure.
What do you think is the correct solution to problem?
Allow $recursiveRef to support $id referencing.
For convenience, the following is reference data I've been using to test.
{
"nodeId": "1",
"nodes": [
{ "nodeId": "2", "nodes": [] },
{ "nodeId": "3", "nodes": [] },
{ "nodeId": "4", "nodes": [] },
{ "nodeId": "5", "nodes": [] }
],
"element": {
"elementId": "3",
"elements": [
{ "elementId": "4", "elements": [] },
{ "elementId": "5", "elements": [] }
]
}
}
It should be possible to copy the above schema and the above data as is into this site to compare.
Yes, it’s a current limitation that requires a bit of thinking to overcome…
@epoberezkin Hey, thanks for the reply. Just as a follow up, I have since tried to replace the $recursiveRef with $ref (which is permissive of arbitrary JSON pointers) and have been able to cause AJV to go into a recursive tailspin. The following analogs the above schema using $defs and $ref.
{
"$id": "Node",
"$ref": "Node#/$defs/self",
"$defs": {
"self": {
"additionalProperties": false,
"type": "object",
"properties": {
"nodeId": {
"type": "string"
},
"nodes": {
"type": "array",
"items": {
"$ref": "Node#/$defs/self"
}
},
"element": {
"$id": "Element",
"$ref": "Element#/$defs/self",
"$defs": {
"self": {
"additionalProperties": false,
"type": "object",
"properties": {
"elementId": {
"type": "string"
},
"elements": {
"type": "array",
"items": {
"$ref": "Element#/$defs/self"
}
}
},
"required": [
"elementId",
"elements"
]
}
}
}
},
"required": [
"nodeId",
"nodes",
"element"
]
}
}
}
Resulting in the following
... omitted
at Ajv2019.getJsonPointer (target\spec\index.js:4153:29)
at Ajv2019.resolveSchema (target\spec\index.js:4103:31)
at Ajv2019.resolveSchema (target\spec\index.js:4108:35)
at Ajv2019.getJsonPointer (target\spec\index.js:4153:29)
at Ajv2019.resolveSchema (target\spec\index.js:4103:31)
at Ajv2019.resolveSchema (target\spec\index.js:4108:35)
at Ajv2019.getJsonPointer (target\spec\index.js:4153:29)
at Ajv2019.resolveSchema (target\spec\index.js:4103:31)
at Ajv2019.resolveSchema (target\spec\index.js:4108:35)
at Ajv2019.getJsonPointer (target\spec\index.js:4153:29)
at Ajv2019.resolveSchema (target\spec\index.js:4103:31)
at Ajv2019.resolveSchema (target\spec\index.js:4108:35)
at Ajv2019.getJsonPointer (target\spec\index.js:4153:29)
at Ajv2019.resolveSchema (target\spec\index.js:4103:31)
at Ajv2019.resolveSchema (target\spec\index.js:4108:35)
... omitted
at CodeGen.code (target\spec\index.js:907:11)
at CodeGen.func (target\spec\index.js:1034:16)
at validateFunction (target\spec\index.js:3447:13)
at topSchemaObjCode (target\spec\index.js:3472:7)
at Object.validateFunctionCode (target\spec\index.js:3432:11)
at Ajv2019.compileSchema (target\spec\index.js:4018:20)
at Ajv2019._compileSchemaEnv (target\spec\index.js:4627:35)
Couldn't get this one to run across other JSON Schema validator either (it has problems resolving the subschema JSON pointer reference for unknown reasons). It looks like AJV is able to resolve it, but seems to be getting into a loop resolving the same pointer over and over during the codegen pass.
I'm guessing $recursiveRef uses a similar resolver code path (hence the # imposed limitation on the pointer). I'm wondering if it's possible to cache the resolved schema via JSON pointer, or if there are further downstream considerations with the validation sharing the same validation / fragment.
-
you only need to use recursiveRef and recursiveAnchor only if you need these schemas to be extensible, in normal cases you can do it with normal refs.
-
you can do what you need by splitting element schema to a separate schema file
-
consider using dynamicRef instead.
@epoberezkin Hey, thanks for the response. I'm not familiar with $dynamicRef but will take a look.
you can do what you need by splitting element schema to a separate schema file
Would this involve moving the recursive sub schema out into it's own schema and adding it to AJV via ajv.addSchema(...)? Would referencing the schema this way help avoid the infinite loop compiling recursive schemas of this type?
Would this involve moving the recursive sub schema out into it's own schema and adding it to AJV via ajv.addSchema(...)?
Yes, with addSchema or via schemas option.
Would referencing the schema this way help avoid the infinite loop compiling recursive schemas of this type?
Splitting and infinite loop is really unrelated, but it would help overcome the limitation. In general, you should not need $recursiveRef - normal $ref should allow creating recursive schemas - there are many examples in the tests.
@epoberezkin Hi, was just looking into this issue again, and seem to be having some success using the 2020-12 draft, replacing $recursiveAnchor and $recursiveRef for $dynamicAnchor and $dynamicRef respectively. The previous JSON test case written for this issue appears to be working great.
import Ajv from 'ajv/dist/2020'
const ajv = new Ajv({})
const Node = {
'additionalProperties': false,
'$id': 'Node', // <------------+
'$dynamicAnchor': 'node', // |
'type': 'object', // | Array of Node
'properties': { // |
'nodeId': { // |
'type': 'string' // |
}, // |
'nodes': { // |
'type': 'array', // |
'items': { // |
'$dynamicRef': '#node' // -------------+
}
},
'element': {
'additionalProperties': false,
'$id': 'Element', // <------------+
'$dynamicAnchor': 'element', // |
'type': 'object', // | Array of Element
'properties': { // |
'elementId': { // |
'type': 'string' // |
}, // |
'elements': { // |
'type': 'array', // |
'items': { // |
'$dynamicRef': '#element' // -------------+
}
}
}
}
}
}
const ok = ajv.validate(Node, {
'nodeId': '1',
'nodes': [
{ 'nodeId': '2', 'nodes': [] },
{ 'nodeId': '3', 'nodes': [] },
{ 'nodeId': '4', 'nodes': [] },
{ 'nodeId': '5', 'nodes': [] }
],
'element': {
'elementId': '3',
'elements': [
{ 'elementId': '4', 'elements': [] },
{ 'elementId': '5', 'elements': [] }
]
}
}) // -> ok
console.log('ok', ok)
This issue was originally raised as AJV was stack overflowing in the $recursive case (possibly tied to the AJV's compilation logic) but seems to be fine for the $dynamic implementation on the 2020 draft. So was curious if there had been updates to AJV over the past 10 months to resolve the stack overflow issue.
Would be happy to close off this issue, but wanted to check before swapping over to the 2020 spec.