ajv icon indicating copy to clipboard operation
ajv copied to clipboard

strictTuples with additionalItems -> strictArrays

Open jscheid opened this issue 4 years ago • 20 comments

What version of Ajv are you using? Does the issue happen if you use the latest version?

7.0.3

Ajv options object

{}

JSON Schema

{
  "type": "array",
  "items": [
    {
      "type": "number"
    }
  ],
  "minItems": 1,
  "additionalItems": {
    "type": "string"
  }
}

Sample data

N/A

Your code

new (require("ajv").default)().compile({ type: "array", items: [{type: "number"}], minItems: 1, additionalItems: {type: "string"} })

Validation result, data AFTER validation, error messages

Warning printed during compilation:

strict mode: "items" is 1-tuple, but minItems or maxItems/additionalItems are not specified or different

What results did you expect?

No warning message.

I'm trying to express the schema "an array with at least one element, which must be a number, followed by any number of strings". Is this the correct way to declare it?

Assuming that it is, I understand that I can turn off this warning with strictTuples but it seems like a case that shouldn't emit a warning, or am I misunderstanding the purpose of strictTuples?

Are you going to resolve the issue?

I suppose I could try my hand at creating a PR. Should this

https://github.com/ajv-validator/ajv/blob/ca2ae61c489f45fa2ec3ff2ee78b10136cb1ed3c/lib/vocabularies/applicator/items.ts#L68-L70

perhaps read something like

function fullTupleSchema(len: number, sch: any): boolean {
  return len === sch.minItems && (len === sch.maxItems || sch.additionalItems !== undefined)
}

?

jscheid avatar Jan 29 '21 22:01 jscheid

The warning is correct.

The idea of this warning is to prevent two common schema mistakes:

  1. schema for a tuple (a data structure with a fixed number of heterogenous elements) when schema author forgot to include restrictions on array size. It is worth noting, that it is recommended not to use tuples in JSON at all, as some languages do not support such data structures well (and JSON is a cross platform format).
  2. schema for array when schema author accidentally wrapped schema that were intended for all items into array, making it a schema for the first item only. It is a very common mistake - there are quite a few issues in Ajv saying "my schema only validates the first item.

The data structure you want to validate is neither homogenous array nor a tuple, so it is better to avoid using it in JSON. If you do need to use it, you should disable strictTuples option (strictTuples: false).

epoberezkin avatar Jan 31 '21 19:01 epoberezkin

Having thought for a minute, it may be worth allowing such schemas in strict mode - it is just strictTuples wouldn't be correct here, as it is not a tuple...

Possibly, in v8 strictTuples can be replaced with strictArrays to allow such schemas as above - need to think about it.

epoberezkin avatar Jan 31 '21 19:01 epoberezkin

Admittedly what really prompted me to open this ticket was not heterogeneous arrays like in my example, but homogeneous ones that look like so:

{
  "type": "array",
  "items": [
    {
      "type": "number"
    }
  ],
  "minItems": 1,
  "additionalItems": {
    "type": "number"
  }
}

Which is what ts-json-schema-generator currently generates for the TypeScript idiom [number, ...number[]].

I'm also having second thoughts, maybe this is better to fix in that other package, it could generate a cleaner schema here.

jscheid avatar Jan 31 '21 20:01 jscheid

I see - interesting... it’s not either or though - it can be both fix there and the change here. I just don’t like the idea to allow non-tuples with strictTuples option (plus it’s a breaking change anyway), but replacing it with strictArray that would allow additionalItems: schema as well, not only false, is probably a good idea - it still solves the original problem of avoiding both types of mistakes and allows a bit more flexible schemas at the same time.

epoberezkin avatar Jan 31 '21 22:01 epoberezkin

also “tuples” isn’t really a term used in JS

epoberezkin avatar Jan 31 '21 22:01 epoberezkin

@epoberezkin I am pretty new with schema/ajv, but how would you make a tuple with one extra optional item (but not more) ? I am getting strict mode: "prefixItems" is 3-tuple, but minItems or maxItems/items are not specified or different at path "#/properties/group" for

{
      type: 'array',
      minItems: 2,
      maxItems: 3,
      items: false,
      prefixItems: [ (3 objects) ]
}

I want exactly one optional extra item, not some. I guess I could disable the warning to solve this...

nekdolan avatar Sep 06 '21 19:09 nekdolan

It’s quite an opinionated feature only allowing tuples with a fixed number of items, which are, arguably, more common… You just need to disable this warning using an option strictTuples: false in your case.

epoberezkin avatar Sep 06 '21 19:09 epoberezkin

I just stumbled on this too. I have a schema like:

{
  type: 'array',
  prefixItems: [{ /* one object to validate against the very first item */ }],
  items: {}, // another object to validate all other items
  minItems: 1,
  maxItems: 6,
}

and this gives me strict mode: "prefixItems" is 1-tuple, but minItems or maxItems/items are not specified or different. So the only way is to disable strict tuples?...

1valdis avatar Nov 22 '21 15:11 1valdis

So the only way is to disable strict tuples?...

Yes, the idea of strictTuples was to discourage the usage of flexible size heterogenous arrays in your message schemas, as they rarely map well to type systems in many languages. If you do need to use them, you can just disable this option (and it's a warning anyway).

epoberezkin avatar Nov 22 '21 19:11 epoberezkin

Not sure if this is related. I'm struggle trying a tuple working in ajv with typescript types.

Types of property 'items' are incompatible.

import { JSONSchemaType } from "ajv"

type Coordinates = [number, number]

type Position = {
    coords?: Coordinates
}

const schema: JSONSchemaType<Position> = {
    type: "object",
    properties: {
        coords: {
            items: {
                type: "number"
            },
            maxItems: 2,
            minItems: 2,
            nullable: true,
            type: "array"
        }
    }
}

Typescript playground link here

reggi avatar Jan 22 '22 05:01 reggi

I came up with the solution from the docs playground here

https://ajv.js.org/json-schema.html#additionalitems

import { JSONSchemaType } from "ajv"

type Coordinates = [number, number]

type Position = {
    coords?: Coordinates
}

const schema: JSONSchemaType<Position> = {
    type: "object",
    properties: {
        coords: {
            additionalItems: false,
            items: [{ type: "integer" }, { type: "integer" }],
            minItems: 2,
            nullable: true,
            type: "array"
        }
    }
}

reggi avatar Jan 22 '22 05:01 reggi

What results did you expect?

No warning message.

Can we have a silent mode or e.g. that the ajv validator throws? I didn't ask for warnings in the console and I'd prefer to not have any either.

TimDaub avatar Sep 16 '22 15:09 TimDaub

Hi all,

I am new using Ajv and I am wondering why in this following type, Ajv ask me to add minItems and maxItems:

export type Waypoints = [Waypoints, Waypoints, ...Waypoints[]];

If I remove maxItems, I get the following error:

 Property 'maxItems' is missing in type '{ type: "array"; minItems: number; items: [{ type: "array"; items: { type: "number"; }; }, { type: "array"; items: { type: "number"; }; }]; additionalItems: { type: string; items: { type: string; }; }; }' but required in type '{ maxItems: number; }'.ts(2322)
json-schema.d.ts(43, 5): 'maxItems' is declared here.

Not sure why this is happening.

guimochila avatar Sep 29 '22 08:09 guimochila

because by default ajv expects you to either use either arrays or fixed size tuples. I am not sure that you want to be using export type Waypoints = [Waypoints, Waypoints, ...Waypoints[]]; - type conversion utility won't be able to cope with it in any case - why not just Waypoints[][]?

epoberezkin avatar Sep 29 '22 09:09 epoberezkin

I what you want to achieve is the array that has at least two items, then you can achieve it on the type level as you did, and you can achieve it in schema with minItems, but type conversion utility won't be able to translate one to another - you can just use Waypoints[][] in the type you pass to the schema and add minItems to the schema, and use more strict type on the applications side...

A better idea could be to use a record something like {point1: ... , point2: ... , extraPoints: ...}, as almost no language allows to have "minimum two items" restriction on the type level, and those that do would have to define a non-standard type for that. Record is a more cross-platform way for that (if you care about that at all).

epoberezkin avatar Sep 29 '22 09:09 epoberezkin

@epoberezkin thanks for the prompt response. I made a mistake on the example code, the export is the following one:

export type Waypoints = [Waypoint, Waypoint, ...Waypoint[]];

Wapoints is an array of Waypoint(singular), however the array must contain at least two items(origin, destination). To make a bit more complex, Waypoint can be an array of numbers(coordinates), ex: [2.233, 2.2333] or it can be an object with some defined properties. I have been struggling to find a way to use Ajv with this type definition. The final goal is to validate an array that contains at least 2 items and the items can be anyOf array of numbers or an object.

guimochila avatar Sep 29 '22 11:09 guimochila

Got it, that’s what I thought in the end. This type invariant (an array with at least two items) is not representable in most type systems, so you may want to reconsider and to use a record with origin and destinations fields and optional waypoints.

Typescript and JSON schema do allow to express this type invariant, but the automatic mapping between typescript and JSON schema does not support it.

epoberezkin avatar Sep 29 '22 11:09 epoberezkin

… Given that JSON schema is cross platform spec, the design decision for strict mode was to prohibit this type invariant and to allow either arrays of variable size (in which case the type would be Waypoint[]) or tuples of fixed size, but not variable size tuples.

you schema should have a schema as the value for items; not an array of schemas if you choose to go with the array. Or you can disable strictTuples and add restriction to the schema but it won’t map to typescript type.

epoberezkin avatar Sep 29 '22 12:09 epoberezkin

Personally, I prefer using JTD spec as it is much simpler and maps to type system of most languages, not only TypeScript, and it’s also an RFC

epoberezkin avatar Sep 29 '22 12:09 epoberezkin

Not sure if the team will agree to move to JTD spec as the project is already using heavily typescript. Just a quick question, does tuples with minItems and maxItems would work? Because checking the requirements the size should be at least 2 and max 150, adding that helped to fix errors from typescript. I am just wondering how to implement an array with two different types. @epoberezkin do you know where I can find examples of array that accepts two different types?

guimochila avatar Sep 29 '22 12:09 guimochila

Is it expected that strictTuples throw the warning in the following configuration?

I am validating an array of arbitrary lengths. Each item in the array is either a tuple of one item that is a string or a tuple of 2 items that are a string and an object.

For instance:

[
  ["tupleOfOne"],
  ["tupleOfTwo", {}]
]

I am using the following schema to validate that JSON file (using anyOf to model discriminated unions based on the number of items in the tuple:

{
  type: 'array',
  minItems: 1,
  items: {
    type: 'array',
    allOf: [
      {
        if: {
          minItems: 1,
          maxItems: 1,
        },
        then: {
          additionalItems: false,
          items: [{ type: 'string', enum: ['a', 'b', 'c'] }],
        },
      },
      {
        if: {
          minItems: 2,
          maxItems: 2,
        },
        then: {
          additionalItems: false,
          items: [{ type: 'string', enum: ['a', 'b', 'c'] }, { type: 'object' }],
        },
      },
    ],
  },
};

Which seems to be draft-07 compliant and validates my input correctly. However, Ajv still throws the warning and the only way for me to get rid of it is to use the following schema:

{
  type: 'array',
  minItems: 1,
  items: {
    type: 'array',
    allOf: [
      {
        if: {
          minItems: 1,
          maxItems: 1,
        },
        then: {
          minItems: 1,
          maxItems: 1,
          additionalItems: false,
          items: [{ type: 'string', enum: ['a', 'b', 'c'] }],
        },
      },
      {
        if: {
          minItems: 2,
          maxItems: 2,
        },
        then: {
          minItems: 2,
          maxItems: 2,
          additionalItems: false,
          items: [{ type: 'string', enum: ['a', 'b', 'c'] }, { type: 'object' }],
        },
      },
    ],
  },
};

It seems that adding the extraneous minItems and maxItems is redundant in that case as we have the guarantee that the schema would only match according to the tuple length.

olivier-martin-sf avatar Feb 15 '23 14:02 olivier-martin-sf