Consider using ajv instead of typebox for schema validation

[ayZagen]

trafficstars

Currently typebox is used for schema validation and I can guess it is chosen for great performance and awesome TypeScript support.

However, In real world applications, following points are mostly necessary with JSON schemas:

• Precise validation errors Typebox really lacks in this area. For example, Union schema errors.

• Conforming defaults (default parameter in json schema) In Typebox we must use Default function

• Removing additional properties ( additionalProperties: false ) In Typebox we must use Clean function

To conform all above points with typebox you must do following:

import { Clean, Default } from "@sinclair/typebox/value"

// assuming `Schema` is compiled with TypeCompiler
if( !Schema.Check( Default( RawSchema, Clean(RawSchema, value) ) ){
  const errors = [...Schema.Error(value)]
}

In ajv you can pass them as options.

Now when you run benchmarks you will see ajv is more performant than typebox.

We can still use TypeBox as schema definitions and pass them to ajv internally. End-Users will not notice that much difference.

ajv is not a new guy in the town and it is used internally by fastify.

Here is a simple benchmark script:

bench.ts

import { Type } from "@sinclair/typebox"
import { TypeCompiler } from "@sinclair/typebox/compiler"
import { Clean, Default } from "@sinclair/typebox/value"
import Ajv from "ajv"
import ajvErrors from "ajv-errors"
import { bench, group, run } from "mitata"

const RawSchema = Type.Object({
  type: Type.Literal("email"),
  name: Type.String(),
  details: Type.Object(
    {
      from: Type.String({
        default: "mydefault",
        maxLength: 512,
      }),
      subject: Type.String({
        maxLength: 512,
      }),
    },
    { default: {} },
  ),
})
const ajv = ajvErrors( new Ajv({
    strict: true,
    strictRequired: false,
    removeAdditional: true,
    useDefaults: true,
    allErrors: true,
    allowUnionTypes: true,
    $data: true,
    discriminator: true
  }),
)
const TypeboxValidator = TypeCompiler.Compile(RawSchema)
const AjvValidator = ajv.compile(RawSchema)

group("json-schema", () => {
  bench("typebox", () => {
    const val = {}
    if (!TypeboxValidator.Check(Default(RawSchema, Clean(RawSchema, val)))) {
      const errors = [...TypeboxValidator.Errors(val)]
    }
  })
  bench("ajv", async () => {
    const val = {}
    const result = AjvValidator(val)
    if (!result) {
      const errors = AjvValidator.errors
    }
  })
})

const ajvPkg = await import("ajv/package.json")
const typeboxPkg = await import("./node_modules/@sinclair/typebox/package.json")

console.log(`ajv: ${ajvPkg.version}`)
console.log(`TypeBox: ${typeboxPkg.version}`)

await run({
  percentiles: false,
})

And in my local potato machine this is the result:

$ bun run bench.ts 
ajv: 8.17.1
TypeBox: 0.33.5
cpu: Intel(R) Core(TM) i7-7700HQ CPU @ 2.80GHz
runtime: bun 1.1.24 (x64-linux)

benchmark      time (avg)             (min … max)
-------------------------------------------------
• json-schema
-------------------------------------------------
typebox     3'260 ns/iter   (1'833 ns … 5'491 µs)
ajv           296 ns/iter     (211 ns … 2'261 ns)

summary for json-schema
  ajv
   11.01x faster than typebox

@SaltyAom I really hope to hear your thoughts about this matter as we are in stage of refactoring our application.

[h3ruta]

It appears that recent TypeBox updates have significantly narrowed the performance gap.

$ bun add @sinclair/[email protected] # the version Elysia uses
$ bun run bench.mts
cpu: unknown
runtime: bun 1.1.24 (arm64-linux)

benchmark      time (avg)             (min … max)
-------------------------------------------------
• json-schema
-------------------------------------------------
typebox     1'291 ns/iter   (1'172 ns … 2'877 ns)
ajv           151 ns/iter       (124 ns … 709 ns)

summary for json-schema
  ajv
   8.55x faster than typebox

$ bun add @sinclair/[email protected]
$ bun run bench.mts
cpu: unknown
runtime: bun 1.1.24 (arm64-linux)

benchmark      time (avg)             (min … max)
-------------------------------------------------
• json-schema
-------------------------------------------------
typebox       298 ns/iter     (247 ns … 1'799 ns)
ajv           146 ns/iter       (119 ns … 696 ns)

summary for json-schema
  ajv
   2.04x faster than typebox

[ayZagen]

@lilprs I dont want this thread to be turn into a benchmarking thread as ajv offers more than typebox. I mentioned those points in the post.

As for benchmark result, it is great to hear typebox improving but the benchmark script I posted above was retrieving only First error for typebox (TypeboxValidator.Errors(val).First())

I have updated the script and the result with latest versions.

[antn9x]

May we have more data validation library for elysia? I'd like to work with ajv too, it's simple and performance.

[BleedingDev]

I second this, well... not ajv, but I'd love to use EffectTS/schema as my whole stack is built around EffectTS. :) I think Elysia would highly benefit from this integration of other parts of the ecosystem. :)

[Extarys]

Is there any way to use ajv with elysia at the moment ? Or is the coupling between elysia, typebox and openapi too tight?

[bertoabist]

Look at this benchmark https://moltar.github.io/typescript-runtime-type-benchmarks/ I think using typebox is correct choice. If you want to select your validation, hono provide alot of options to select.

[ayZagen]

@bertoabist I think you only read issue title and posted a comment. Read the description.

[SaltyAom]

Hi, this is a preview update of 1.3 from #1141

The following are compiled instruction from Elysia

// c.body
let val = {};

const defaultValue = {
	details: {
		from: "mydefault",
	}
};

val = isNotEmpty(val) ? Object.assign(defaultValue, val) : mirror(val);

if (!TypeboxValidator.Check(val)) {
	// Remove as production build doesn't log error
	// const errors = TypeboxValidator.Errors(val).First();
	return false;
}

bench.ts

import Ajv from "ajv";

import { Type } from "@sinclair/typebox";
import { TypeCompiler } from "@sinclair/typebox/compiler";
import { createMirror } from "exact-mirror";

import {
	barplot,
	compact,
	summary,
	run as mitataRun,
	bench as add,
} from "mitata";

export const mitata = (callback: Function) => {
	compact(() => {
		barplot(() => {
			summary(() => {
				callback();
			});
		});
	});

	mitataRun();
};

const RawSchema = Type.Object({
	type: Type.Literal("email"),
	name: Type.String(),
	details: Type.Object(
		{
			from: Type.String({
				default: "mydefault",
				maxLength: 512,
			}),
			subject: Type.String({
				maxLength: 512,
			}),
		},
		{ default: {} },
	),
});

const ajv = new Ajv({
	strict: true,
	strictRequired: false,
	removeAdditional: true,
	useDefaults: true,
	allErrors: true,
	allowUnionTypes: true,
	$data: true,
	discriminator: true,
});

const TypeboxValidator = TypeCompiler.Compile(RawSchema);
const AjvValidator = ajv.compile(RawSchema);
const mirror = createMirror;

const isNotEmpty = (obj?: Object) => {
	if (!obj) return false;

	for (const x in obj) return true;

	return false;
};

mitata(() => {
	// This is equivalent to Elysia compile
	add("typebox", () => {
		// c.body
		let val: any = {};

		const defaultValue = {
			details: {
				from: "mydefault",
			},
		};

		val = isNotEmpty(val) ? Object.assign(defaultValue, val) : mirror(val);

		if (!TypeboxValidator.Check(val)) {
			// Remove as production build doesn't log error
			// const errors = [...TypeboxValidator.Errors(val)];
			return false;
		}
	});

	// Equivalent to typebox
	add("ajv", async () => {
		const val = {};
		const result = AjvValidator(val);
		if (!result) {
			return false;
		}
	});
});

What happend and why?

1. Value.Default is precompile into a literal value, which produced a const defaultValue
2. Value.Clean is replaced with exact-mirror which is significantly faster
3. On production server, TypeBoxValidator.Errors is never called, so it's omitted from benchmark

Here's the result:

benchmark                   avg (min … max) p75 / p99    (min … top 1%)
------------------------------------------- -------------------------------
typebox                      142.19 ns/iter 136.95 ns 273.97 ns ▇█▂▁▁▁▁▁▁▁▁
ajv                          128.17 ns/iter 125.73 ns 254.90 ns █▇▁▁▁▁▁▁▁▁▁

                             ┌                                            ┐
                     typebox ┤■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■■ 142.19 ns
                         ajv ┤ 128.17 ns
                             └                                            ┘

summary
  ajv
   1.11x faster than typebox

Although it's a little bit slower but this shouldn't be a performance concern as there are likely to be other area that would worth an effort to improve instead of optimizing for a few nanoseconds.

This is available on Elysia 1.3 stable and can be preview since around 1.3.0-exp.10.

[SaltyAom]

Closing this as not planned

As we don't have any significant reason to switch to ajv (at least yet) The performance gap isn't a big of a concern with recent version of TypeBox, and our built-in Exact Mirror