json-schema-spec icon indicating copy to clipboard operation
json-schema-spec copied to clipboard

Request: Add precision and scale keywords for numeric instances

Open SrikrishnanS opened this issue 3 years ago • 9 comments

We often see use cases, where numeric values should have exactly n digits after decimal point. Example currency value should have 2 decimal places only: {"price": 17.23}

Suggestion is to include precision and scale keywords in the spec. Precision is defined as number of digits in a number and scale is number of digits after decimal point.

I understand that scale can be achieved using multipleOf keyword, i.e. "multipleOf" : 0.01 ensures scale value 2, and "maximum": 999999 ensures precision value 6.

However precision and scale keywords should be meaningful and much cleaner.

SrikrishnanS avatar Sep 03 '21 00:09 SrikrishnanS

"number of digits in a number" can be handled by minimum and maximum, but I could see a use for the minimum and maximum number of digits after the decimal point (and is not too difficult to implement).

karenetheridge avatar Sep 03 '21 18:09 karenetheridge

Great. Can you include scale keyword in the spec? Does it have to be okay'ed by a wider audience?

SrikrishnanS avatar Sep 07 '21 02:09 SrikrishnanS

It occurs to me that the number type is not sufficient to express these types of constraints and what you probably want is a string with a pattern or a custom format value. Remember that JSON Schema works on the JSON data model, not the text itself, so a number of precision = 2 would require something like 1.00, but once that JSON string is parsed to JSON data, there is no way to tell that those two zeros were included in the JSON. Scale can have a similar problem in that a number with a scale = 6 could be represented in JSON as 000001, but JSON Schema wouldn't know that it had those leading zeros. If you don't care about missing leading or trailing zeros, then the point wasn't really the number of digits but rather the max/min value and the max precision, which I think is better expressed with the current keywords.

jdesrosiers avatar Sep 08 '21 20:09 jdesrosiers

Looks like we have come across a similar question before https://stackoverflow.com/questions/53449740/how-can-i-validate-number-of-digits-in-a-number-using-json-schema-ajv/53449981#comment93772686_53449981

Watch out!

JS treat numbers with leading zeros as octal only if they valid octal, if not then it treat it as decimal.

https://stackoverflow.com/a/37004175/89211

Relequestual avatar Sep 09 '21 10:09 Relequestual

Luckily, the JSON specification prohibits octal numbers in JSON, so we don't have to worry about representation formats.

karenetheridge avatar Sep 09 '21 16:09 karenetheridge

Let me clarify what I had in my mind. By scale, I meant the maximum number of decimal places that can occur in a number value. Applications may not want to use strings to store numbers as it may involve conversions back and forth. In databases, typically, there is a NUMBER type: NUMBER(m, n) where m is precision and n is scale. For string serialization use cases, there are ways to do ensure 3 characters after decimal point. But when stored, apps may not want to exceed n decimal digits...

SrikrishnanS avatar Sep 09 '21 16:09 SrikrishnanS

It might be helpful to provide some examples of these keywords, and some data that does or does not validate against them. For precision especially I do not understand your intent.

karenetheridge avatar Sep 09 '21 17:09 karenetheridge

@karenetheridge Sure. For example, the number 12.56 has a precision of 4 and a scale of 2.

A schema with my proposed keywords:

{
  "type": "number"
  "precision": 4
  "scale": 2
}

This schema accepts numbers that have at most 4 digits in it (including the ones after decimal place). The number of digits after decimal point should not exceed 2.

Invalid values:

89248   --> Exceeds precision
233.23  --> Exceeds precision
9.9995  --> Exceeds precision and scale
0.003   --> Exceeds scale

Valid values:

8423.0000
23
5
2773
23.43
325.5

Of course, the following two schemas are equivalent:

{
  "type": "number",
  "precision": 3
}

and

{
  "type": "number",
  "maximum": 999
}

I see practical use cases for these in JSON document.

SrikrishnanS avatar Sep 09 '21 21:09 SrikrishnanS

In databases, typically, there is a NUMBER type: NUMBER(m, n) where m is precision and n is scale.

Yes, databases have a number type that is capable of internally representing numbers where scale and precision are meaningful. This is not true for most programming languages which use IEEE 754 floating point numbers by default. For example, if you compute 1.1 + 0.6 you would expect 1.7 which has a precision of 2 and scale of 1. But, what you really get is 1.7000000000000002.

I see this as in the same category as representing dates in JSON. There is no Date type in JSON, so you need to represent it as a string and use format to indicate what type that value truly represents. The NUMBER(m, n) type is a fundamentally different type than number and therefore should be represented as a string with a format indicating it's true type. It might be possible for it to be a number with a format. I don't think you would necessarily loose anything if the number was parsed as floating point and then converted to NUMBER(m, n). I think floating point numbers only start to get wonky when you do math on them. Either way, I don't see validation keywords being the right approach for this problem. I think a format (or something of that nature) is a better approach.

jdesrosiers avatar Sep 10 '21 19:09 jdesrosiers