effect

effect copied to clipboard

What version of @effect/schema is running?

0.52.0

What steps can reproduce the bug?

import * as S from "@effect/schema/Schema";
import * as JSONSchema from "@effect/schema/JSONSchema";

const d1 = JSONSchema.to(S.Date)
const d2 = JSONSchema.to(S.DateFromSelf)
const d3 = JSONSchema.to(S.DateFromString);

// they all return Error: cannot build a JSON Schema for declarations without a JSON Schema annotation

// but this will not work either
JSONSchema.to(S.Date).pipe(S.jsonSchema({type: "string", format: "date-time"}))

EDIT I think the problem / my misconception can be more generalized. I would expect the explicit definition to override built-in behavior, so the JSON schema of this should be a number, but is a string: S.string.pipe(S.jsonSchema({ type: "number" }))

What is the expected behavior?

While the correct representation of Dates in JSON schema is arguable, I would expect that S.DateFromString.pipe(S.jsonSchema({type: "string", format: "date-time"})) allows me to override the JSON schema annotation and fix the above error, but to no avail.

What do you see instead?

Error: cannot build a JSON Schema for declarations without a JSON Schema annotation

Additional information

When you use JSONSchema.to with S.Date

const jsonSchema = JSONSchema.to(S.Date) // throws

you are trying to build a JSON Schema for the type type To = S.Schema.To<typeof S.Date> which is Date, and that's impossible.

@gcanti Sorry, i just edited the issue, while you were posting. My problem in general (or my misconception) is, that S.jsonSchema does not override.

S.string.pipe(S.jsonSchema({ type: "number" }))

I would expect the explicit definition to override built-in behavior, so the JSON schema of this should be a number, but is a string

Why you would do that though?

import * as S from "@effect/schema/Schema"
import * as JSONSchema from "@effect/schema/JSONSchema"

console.log(JSON.stringify(JSONSchema.to(S.string.pipe(S.jsonSchema({ type: "number" }))), null, 2))
/*
{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "string",
  "description": "a string",
  "title": "string"
}
*/

here

{
  "$schema": "http://json-schema.org/draft-07/schema#",
  "type": "string",
  "description": "a string",
  "title": "string"
}

is the correct representation of S.string.

Adding a json schema annotation makes sense when you are defining a refinement.

My use case is, I have a property that is a date or a date, that has been JSON serialized (ISO string). This property is used on the backend layer, where we already have a Date object, and on the frontend side, where we also have Date objects, but due to JSON as network protocol, it gets transformed to an ISO string.

import * as S from "@effect/schema/Schema";

export const DateSchema = S.union(S.Date, S.DateFromSelf).pipe(
  S.jsonSchema({
    description: "A valid date as an ISO 8601 formatted string.",
    format: "date-time",
    title: "ISO 8601 date string",
    type: "string",
  }),
);

Or asked differently, how would a Schema look like that describes a ISO-date string, from which a correct JSON schema of { type: "string", format: "date-time"} can be inferred?

Regarding why I would do that: It's a generalized example. I just feels weird, that S.jsonSchema({...}) does not allow overriding. I mean you can also construct something like S.string.pipe(S.filter((i) => typeof i === "number"))

I mean you can also construct something like S.string.pipe(S.filter((i) => typeof i === "number"))

Yeah, exactly, and in that case you can add a json schema annotation as I said above since it's a refinement:

import * as S from "@effect/schema/Schema"
import * as JSONSchema from "@effect/schema/JSONSchema"

const schema = S.string.pipe(
  S.filter((i) => typeof i === "number", {
    jsonSchema: { type: "number" }
  })
)

console.log(JSONSchema.to(schema))
/*
{
  '$schema': 'http://json-schema.org/draft-07/schema#',
  type: 'number',
  description: 'a string',
  title: 'string'
}
*/

which is super weird in this particular case, but it works nonetheless.

how would a Schema look like that describes a ISO-date string, from which a correct JSON schema of { type: "string", format: "date-time"} can be inferred?

Something like

import * as S from "@effect/schema/Schema"
import * as JSONSchema from "@effect/schema/JSONSchema"
import * as Ajv from "ajv"
import addFormats from "ajv-formats"

const ISOString = S.string.pipe(
  S.filter((s) => s === new Date(s).toISOString(), {
    description: "an ISO-date string",
    title: "ISOString",
    jsonSchema: { format: "date-time" }
  })
)

const ajv = new Ajv.default()
addFormats(ajv)

console.log(ajv.compile({ type: "string", format: "date-time" })(new Date().toISOString())) // true

const jsonSchema = JSONSchema.to(ISOString)

console.log(jsonSchema)
/*
{
  '$schema': 'http://json-schema.org/draft-07/schema#',
  type: 'string',
  description: 'an ISO-date string',
  title: 'ISOString',
  format: 'date-time'
}
*/

console.log(ajv.compile(jsonSchema)(new Date().toISOString())) // true

Okay, thank you. I understand the direction. As the goal of the package is gathering feedback: I don't understand why S.jsonSchema is exported at all. When would it be used? I thought S.string.pipe(S.jsonSchema({ type: "date-string"})) to be valid (although probably a bad idea, since Json schema and schema are now out of sync). But apparently it is not and you would use jsonSchema annotations only inside a refinement.

I would always favor developer experience, so I think it makes sense to allow devs to short-circuit the JSON schema (the same way you can with the weird string / number example). Also I think it is transparent and easy to reason about (Sh** in / Sh** out). But I understand if maintainers want to put tighter guardrails in place to avoid drift between Schema and JSON schema.

Another use case I have for overrides: We rely heavily on JSON schema for various tooling. I have a fairly large schema due to a single property in an object. The resulting schema is too large for one of our third party tools. For me a very ergonomic way would be to override / short circuit the JSON schema for the given property.

Another use case I have for overrides...

That's a valid scenario, thanks for the feedback.

Also I think it is transparent and easy to reason about (Sh** in / Sh** out)

On the whole, I agree with you. While it might introduce inconsistent behavior, it also empowers end users to accomplish what they need without unexpected built-in behaviors.

Ok I took a look at the code and I have encountered a few uncertainties. I propose dividing the problem into two distinct parts:

1. (current state) add documentation and checks to ensure that JSON Schema annotations can be exclusively applied to refinements
2. (future consideration) investigate the possibility of enabling overrides (and how to do so without interfering with refinements)

Regarding 2), I'm uncertain about when and how it's worthwhile to apply these overrides. Could you please provide one or more real examples of your current use cases?

Just encountered a situation today with AWS api gateway (which needs JSON schema draft 4, so we wrote our custom version anyway), where AWS couldn't handle circular deps in the model, so I had to alter the JSON schema.

I believe use cases will boil down to integration with other tooling. We sometimes need to "massage" the schema a bit here and there. More often than not, this introduces drift that is manageable and acceptable, especially since the different JSON schemas are generated derivates of the effect schema. In the AWS scenario, my effect schema is still the source of truth, but with overrides, I can integrate the schema with different tools, which unfortunately have their own quirks that I need to work around, but that's a reality. Hope this makes sense.

Why would you want to make sure that jsonSchema is not interfering with refinements? I would say the most straightforward solution would be for the AST walker function to first check whether a JSON schema annotation exists and – if so – to return it.