Add Zod and Effect Schema support for better DX

[gunta]

Is your feature request related to a problem? Please describe. The whole thing of defining a schema in a JSON file its a bit outdated compared to modern solutions.

Describe the solution you'd like We are talking about something along the lines of [Kubb's Infer], Typia, Zod and Effect Schema.

This would definitely improve the DX as a whole for TypeScript developers.

[jamesgpearce]

Did you see this? https://tinybase.org/guides/schemas-and-persistence/schema-based-typing/

When you use the 'with-schemas' versions of the definitions, you get inferred APIs based on the JSON.

Let me know if that is what you had in mind or how it could be improved!

[gunta]

Ok, I'll try that one! Thanks

[jamesgpearce]

Assuming that's an OK approach, can I close out this issue?

[gunta]

Ok! It worked :)

However I was thinking of making the schemas more "modern" and "common" along most used practices around the corner.

There are multiple ways to write a schema in TS, and it may be a good idea to support multiple ways since it is common for the developer to prefer the syntax that one is already using in the project.

For instance, Drizzle supports three different schema tastes:

• Zod schemas
• Typebox schemas
• Valibot schemas

So it might make sense to support all, or some of them.

Also, considering that TinyBase schema types are very simple, perhaps doing the schema typing as Typia or Deepkit might work well enough.

So instead of having this:

const tablesSchema = {
  pets: {
    species: {type: 'string'}
    sold: {type: 'boolean'}
    total: {type: 'number'}
  },
} as const;

We could just have:

const tablesSchema = {
  pets: {
    species: string
    sold: boolean 
    total: number
  },
} as const;

[MentalGear]

Providing zod integration would be much appreciated. Not only is it probably the most feature-rich and most popular validation framework, but many automatic FOSS form kits (e.g superforms, formsnap) exist that make it easy to use the same schema seamlessly on the front- and backend, streamlining the whole application code and cutting overhead tremendously.

[gunta]

My new stake is: Add schema adapters for the following libraries

1. Zod

• The mainstream one
• Everyone knows how to use it
• Simple
• Zero learning curve

2. Effect Schema

• The hottest one
• The most feature rich
• The one used the most in Local First libraries (for a reason):
  • LiveStore
  • ElectricSQL
  • DXOS
  • Jazz
  • Automerge
  • Evolu

Social proof: https://x.com/schickling/status/1761707815016559048

Since Effect Schema has became the defacto standard for the data layer in LoFi libraries, I believe it to be a no-brainer to add support for it.

Star History

Effect 3.0 GA has just been released so its getting traction.

[jamesgpearce]

OK, I think we could have a go at this, though of course it will be a limited dialect of what most of these other schemas are capable of. One question is whether this should be a dev-/build-time process (to create a TinyBase schema from these others) or something that can just be handled at runtime. I need to get familiar with all of these and what common pattern might work. (Hence I'm going to say it'll be after 5.0...!)

[gunta]

One question is whether this should be a dev-/build-time process (to create a TinyBase schema from these others) or something that can just be handled at runtime. I need to get familiar with all of these and what common pattern might work.

Looking at how other libraries solve the problem might be a hint: Drizzle creates a runtime extension for each schema validation library.

https://github.com/drizzle-team/drizzle-orm/tree/main/drizzle-zod https://github.com/drizzle-team/drizzle-orm/tree/main/drizzle-typebox

If performance becomes an issue, a dev/build time process can be added later.

[jamesgpearce]

I am going to focus on schemas in 5.1 and 5.2. Hang in there!

[waynesbrain]

+1 for typebox support.

You can see some differences it has with zod in the comments linked here - https://github.com/colinhacks/zod/issues/2482

A typebox integration would also be preferable for anyone building servers with Fastify where typebox is the blessed schema provider.

[skorphil]

Currently I'm using function to convert zod to tiny schema. Might be helpful as a workaround. But it has many limitations.

import type { CellSchema } from "tinybase/with-schemas";
import type { ZodType as ZodSchema, ZodFirstPartySchemaTypes } from "zod";
import z from "zod";

export function zObjectToTinyTable(
  zodSchema: z.ZodObject<Record<string, ZodSchema>>
) {
  const keys = zodSchema.keyof().options;
  const rows: Record<string, CellSchema> = {};
  keys.forEach((key) => {
    const typeString = zodTypeString(zodSchema.shape[key]);
    rows[key] = { type: typeString };
  });

  return rows;
}

function zodTypeString(zodSchema: ZodSchema): "string" | "number" | "boolean" {
  const typedSchema = zodSchema as ZodFirstPartySchemaTypes;
  switch (typedSchema._def.typeName) {
    case z.ZodFirstPartyTypeKind.ZodString:
      return "string";
    case z.ZodFirstPartyTypeKind.ZodNumber:
      return "number";
    case z.ZodFirstPartyTypeKind.ZodBoolean:
      return "boolean";
    case z.ZodFirstPartyTypeKind.ZodOptional:
      return zodTypeString((zodSchema as z.ZodOptional<ZodSchema>).unwrap());
    case z.ZodFirstPartyTypeKind.ZodEffects:
      return zodTypeString((zodSchema as z.ZodEffects<ZodSchema>).innerType());
    case z.ZodFirstPartyTypeKind.ZodLiteral: {
      const value = typeof typedSchema._def.value;
      if (value === "boolean" || value === "string" || value === "number")
        return value;
      throw Error(`${typedSchema._def.value} not supported by tinyBase`);
    }
    default:
      throw Error(`${typedSchema._def.typeName} not supported by tinyBase`);
  }
}

[bstro]

I've been using Typebox with Tinybase directly without any problems. Simple example:

import { type TablesSchema, createStore } from "tinybase/with-schemas";
import { Type } from "@sinclair/typebox";

const cats = Type.Object({
  name: Type.String(),
  age: Type.Number(),
  breed: Type.String(),
  aloof: Type.Boolean({ default: true })
})

const tablesSchema = {
  cats: cats.properties
} as const satisfies TablesSchema;

const petStore = createStore().setTablesSchema(tablesSchema);

I think this works because each member of someTypeboxSchemaObject.properties has type and default fields that match what Tinybase is looking for.

You'll want to only use the types that Tinybase supports (iirc string, boolean, number).

[joshymcd]

As another bit of info that might help, i use the below type helpers to convert a zod schemas type into a tinybase type (so i get a type error if the two arent in sync )

import { type CellSchema } from "tinybase/with-schemas";

// TypeScript type helpers for converting Zod inferred types to TinyBase schema using actual CellSchema
type InferCellSchema<T> = T extends string | undefined
  ? { type: "string"; default?: string }
  : T extends number | undefined
    ? { type: "number"; default?: number }
    : T extends boolean | undefined
      ? { type: "boolean"; default?: boolean }
      : T extends string
        ? { type: "string"; default?: string }
        : T extends number
          ? { type: "number"; default?: number }
          : T extends boolean
            ? { type: "boolean"; default?: boolean }
            : never;

// Convert Zod inferred type to TinyBase TableSchema (excluding id field)
export type InferTinyBaseSchema<T> = {
  [K in keyof Omit<T, "id">]: InferCellSchema<T[K]>;
};

i can then use it like

import { z } from "zod";
import { type InferTinyBaseSchema } from "@/lib/tinybase-helpers";

// Validation schema for Zod
export const machineSchema = z.object({
  id: z
    .string()
    .optional() ,
  name: z
    .string()
    .min(1, "Name is required") , 
  make: z.string().optional(),
  model: z.string().optional(),
  createdAt: z
    .number()
    .optional(),
  updatedAt: z
    .number()
    .optional()
});

// Generate base TinyBase schema from Zod type (excluding id field) - for type checking only
type MachineTableSchemaBase = InferTinyBaseSchema<Machine>;

export const machineTableSchema: MachineTableSchemaBase = {
  name: { type: "string" as const },
  make: { type: "string" as const, default: undefined }, // Optional field
  model: { type: "string" as const, default: undefined }, // Optional field
  createdAt: { type: "number" as const, default: undefined }, // Unix ms timestamp
  updatedAt: { type: "number" as const, default: undefined }, // Unix ms timestamp
};