zod icon indicating copy to clipboard operation
zod copied to clipboard

[Suggestion] Stricter boolean coerce option

Open pheuter opened this issue 1 year ago • 34 comments

Since we're currently using Svelte Kit with form actions that make heavy use of the FormData browser api, the new Zod 3.20 release is a very much welcome improvement, specifically the new coercion functionality (since all form post values are serialized as strings over the wire).

One suggestion/request I have is to allow boolean coerce to take a strict option that would check for explicit boolean strings:

Current:

const schema = z.coerce.boolean();
schema.parse("true"); // => true
schema.parse("false"); // => true

Suggestion:

const schema = z.coerce.boolean({ strict: true });
schema.parse("true"); // => true
schema.parse("false"); // => false
schema.parse("123"); // => ZodError

Update: If you're using SvelteKit like me, then worth taking a look at sveltekit-superforms. We're using it now and haven't looked back.

pheuter avatar Dec 04 '22 14:12 pheuter

Hm :(

This is definitely an important use case but I think the proposed behavior is pretty weird. It's inconsistent with how coercion is handled with other data types. I don't think z.coerce is good for this particular use case.

Seems like a better approach for the example you mentioned is something like this:

z.string().transform(JSON.stringify).pipe(z.boolean());

I've been thinking about ways to make it easier to parse FormData with Zod for a while but struggling. FormData is so astonishingly bad as to be nearly unsalvagable and I hate it 🤷‍♂️

colinhacks avatar Dec 05 '22 20:12 colinhacks

Perhaps a zod-formdata lib is called for

import * as zfd from "zod-formdata";

zfd.formdata({
  checkbox: zfd.boolean(),
  field: zfd.string(),
  array: zfd.all(z.string()) // uses `FormData.getAll()`,
  jsonified: zfd.json() // input must be string, Zod calls `JSON.parse` automatically
})

IDK this is a very early

colinhacks avatar Dec 05 '22 20:12 colinhacks

I think the proposed behavior is pretty weird.

I haven't used Zod nearly enough to have a good sense of what's idiomatic, so completely defer to your intuitions here, was mainly trying to convey intent and not implementation.

FormData is so astonishingly bad as to be nearly unsalvagable and I hate it 🤷‍♂️

Couldn't agree more :)

Perhaps a zod-formdata lib is called for

Yeah maybe. There's this one but it doesn't play too great with the latest Zod library, would definitely love nothing more than to have the official library handle FormData better. I anticipate this use case becoming only more popular over time as more frameworks embrace progressive enhancement and built-in web apis.

pheuter avatar Dec 05 '22 20:12 pheuter

Handling form submissions server-side using FormData is egregious and I hope Remix moves away from it. FormData isn't actually supported by Node or Bun. I don't understand how people handle forms with nested fields or variable-number fields (e.g. "+ Add another") but I would really miss something like RHF's register("children[0].name") that is able to resolve the form data to a nested object.

Anyway if you're using Remix then Zodix is worth looking into: https://github.com/rileytomasek/zodix

colinhacks avatar Dec 05 '22 20:12 colinhacks

We're actually using Svelte Kit, which has the concept of Form Actions to handle await request.formData() server-side: https://kit.svelte.dev/docs/form-actions

pheuter avatar Dec 05 '22 21:12 pheuter

FormData isn't actually supported by Node or Bun.

Maybe I'm misunderstanding, but the following works for me on Node v19:

// In `node` repl:
> new FormData()

pheuter avatar Dec 05 '22 21:12 pheuter

It is supported since Node 18.0.0 (see https://developer.mozilla.org/en-US/docs/Web/API/FormData)

kibertoad avatar Dec 05 '22 22:12 kibertoad

Ah good catch.

@pheuter want to try to propose an API/behavior that would work for you? I've never gotten very far here. Some open questions:

  • The set of input types that need to be supported
  • Whether to call .get vs .getAll on a given field
  • Whether to JSON.parse the value
  • Special FormData coercion? Should "on" be converted to true?

colinhacks avatar Dec 12 '22 20:12 colinhacks

Some (not exhaustive) thoughts and ideas:

z.formData() as a z.object() analog:

const schema = z.formData({
  channelType: z.enum(["public", "private"]), // formData.get("channelType")
  channelName: z.string().min(3), // formData.get("channelName")
  members: z.array(z.string()) // formData.getAll("members")
})

schema.parse(await request.formData())

I think the trickiest bit here is coercion. Prob can safely ignore File from the FormDataEntryValue union type and just assume .get() => string | null and .getAll() => string[]. This does assume that Zod can handle coercing "true" and "false" to true and false, and more broadly being able to coerce FormData strings into Zod numbers, enums, etc...

pheuter avatar Dec 13 '22 13:12 pheuter

Workaround:

const booleanParamSchema = z.enum(["true", "false"]).transform((value) => value === "true")

denniscalazans avatar Dec 27 '22 15:12 denniscalazans

Hey another workaround from me, treats "0" and "false" as false, "1" and "true" astrue. Defaults to false. Works with string enums since I'm using this with query params in my endpoint. eg. api.com/property=1

// Coerces a string to true if it's "true" or "1", false if "false" or "0"
export const coerceBoolean = z
  .enum(["0", "1", "true", "false"])
  .catch("false")
  .transform((value) => value == "true" || value == "1");

egecavusoglu avatar Jan 19 '23 06:01 egecavusoglu

Ignoring any strong opinions about FormData, supporting it would mean you'd get support for validating URLSearchParams as well, since they are nearly identical interfaces with all the same capabilities and constraints for zod.

and I hope Remix moves away from it

Won't happen. It's the JavaScript object representation of the HTML <form> interface. Remix supports HTML. That means being able to submit forms with client side JavaScript but also letting the browser make the FormData request the way the web has always worked.

<3 zod :)

ryanflorence avatar Jan 24 '23 16:01 ryanflorence

Prob safe to say same for SvelteKit.

pheuter avatar Jan 24 '23 16:01 pheuter

I'm not sure this needs to be zod's job though. Something else can parse the form data and then hand it to zod. FormData can get complicated with multipart.

ryanflorence avatar Jan 24 '23 18:01 ryanflorence

Workaround:

const booleanParamSchema = z.enum(["true", "false"]).transform((value) => value === "true")

I was just writing this very thing, but with the addition of True False and TRUE FALSE because, there are heathens around us.

WORMSS avatar Mar 10 '23 20:03 WORMSS

+1 for a strict boolean check. Workarounds are okay, but having a pretty one-liner for checking "true" and "false" booleans without transforms would be great.

exsesx avatar Mar 15 '23 22:03 exsesx

hey, any plans about this suggestion? It will be very usefull for parsing env variables:

process.env.IS_ENABLED = "false";

// now
const schema = z.coerce.boolean()
const result = schema.parse(process.env.IS_ENABLED) // true

// want
const schema = z.coerce.boolean({ strict: true })
const result = schema.parse(process.env.IS_ENABLED) // false

noveogroup-amorgunov avatar May 18 '23 19:05 noveogroup-amorgunov

Workaround:

z.preprocess((v) => z.enum(['true', 'false']).transform((v) => JSON.parse(v)).catch(v).parse(v),z.boolean())

oljimenez avatar Jun 23 '23 02:06 oljimenez

Here's another workaround that's case-insensitive but expects true/false explicitly or adds an error. (@oljimenez when I paste yours the .catch(v) doesn't seem to exist - at least in my version of zod)

.optional() can be called from this as needed. Hopefully the params passed to addIssue make sense, first time using them 🙂.

export const booleanStrict = z.string().transform<boolean>((v, ctx) => {
  v = v.toLowerCase();
  switch (v) {
    case 'true':
      return true;
    case 'false':
      return false;
    default:
      ctx.addIssue({
        code: z.ZodIssueCode.invalid_type,
        expected: z.ZodParsedType.boolean,
        received: z.ZodParsedType.string,
        message: 'Expected "true" or "false"',
      });
      return false;
  }
});

lzehrung avatar Jul 06 '23 13:07 lzehrung

Here's another workaround that's case-insensitive but expects true/false explicitly or adds an error. (@oljimenez when I paste yours the .catch(v) doesn't seem to exist - at least in my version of zod)

.optional() can be called from this as needed. Hopefully the params passed to addIssue make sense, first time using them 🙂.

export const booleanStrict = z.string().transform<boolean>((v, ctx) => {
  v = v.toLowerCase();
  switch (v) {
    case 'true':
      return true;
    case 'false':
      return false;
    default:
      ctx.addIssue({
        code: z.ZodIssueCode.invalid_type,
        expected: z.ZodParsedType.boolean,
        received: z.ZodParsedType.string,
        message: 'Expected "true" or "false"',
      });
      return false;
  }
});

https://zod.dev/?id=catch

oljimenez avatar Jul 06 '23 13:07 oljimenez

+1 for strict option

chambaz avatar Jul 07 '23 18:07 chambaz

+1 for strict boolean option. If I'm coercing, it should at least have an option to truly transform the value

ronymmoura avatar Aug 06 '23 02:08 ronymmoura

+1 for strict option. it would be very helpful

casder-succ avatar Aug 14 '23 14:08 casder-succ

My workaround. Since I'm validating both on frontend and backend using the same schema, this works for me:

  const booleanSchema = z
    .union([z.boolean(), z.literal('true'), z.literal('false')])
    .transform((value) => value === true || value === 'true')

syahmifauzi avatar Sep 07 '23 17:09 syahmifauzi

Coercion is a difficult thing to nail down for all use cases, but I really think "false" when being coerced into a boolean should be considered false - this is primarily driven by the AJV coercion table

While AJV is far from the first or only library to support schema validation with coercion it is certainly the most widely adopted and I think for primitive coercion they really have hit the mark.

joshuat avatar Oct 06 '23 15:10 joshuat

function envBoolean(params: { optional: boolean; defaultValue: boolean }) {
    type BoolEnum = ['true', 'false'];
    let variable: z.ZodCatch<z.ZodEnum<BoolEnum>> | z.ZodEnum<BoolEnum>;

    if (params.optional) {
        // if undefined assign the defaultValue
        variable = z.enum(['true', 'false']).catch(params.defaultValue ? 'true' : 'false');
    } else {
        // not optional so "true" or "false" is enforced
        variable = z.enum(['true', 'false']);
    }

    // convert string to bool
    return variable.transform((v) => v === 'true');
}

This works for me. I am using this to parse .env

const envSchema= z.object({
  IS_COOL: envBoolean({ optional: true, defaultValue: true })
})

export const env = envSchema.parse(process.env);

LarsOlt avatar Oct 27 '23 11:10 LarsOlt

Finally I reached this issue since I tried to parse environment variables.

As @colinhacks mentioned in https://github.com/colinhacks/zod/issues/1630#issuecomment-1338124864, I agree that strict option is inconsistent with other data types. It is clear thatz.boolean() represents Boolean().

However we also need a utility to transform string to boolean in many use cases. So I suggest a new string transformation: z.string().boolean()

I think the transforming specification is good to be the same with Go standard library's strconv.ParseBool()

https://pkg.go.dev/strconv#ParseBool

ParseBool returns the boolean value represented by the string. It accepts 1, t, T, TRUE, true, True, 0, f, F, FALSE, false, False. Any other value returns an error.

jlandowner avatar Nov 23 '23 16:11 jlandowner

Another option for people to use:

z
    .enum(["true", "false"])
    .nullish()
    .transform((v) => v === "true")

elie222 avatar Dec 08 '23 02:12 elie222

My solution work for all ['true', 'false', true, false] also returns boolean type

const booleans = ['true', 'false', true, false];
const BooleanOrBooleanStringSchema = z
  .any()
  .refine((val) => booleans.includes(val), { message: 'must be boolean' })
  .transform((val) => {
    if (val === 'true' || val === true) return true;
    return false;
  });
type BooleanOrBooleanStringType = z.infer<typeof BooleanOrBooleanStringSchema>;  //boolean

Elalfy74 avatar Dec 13 '23 00:12 Elalfy74

Another option for people to use:

z
    .enum(["true", "false"])
    .nullish()
    .transform((v) => v === "true")

I've found this onefrom @elie222 the most useful for my use case, as it enforces strict-ish string values. I use it to coerce booleans during env var parsing and validations. For reference here is how it works:

expect(coercer.parse("false")).toBe(false);
expect(coercer.parse("true")).toBe(true);
expect(() => coercer.parse(undefined)).toThrow();

For future readers, there is also some degree of advice about this over at https://env.t3.gg/docs/recipes#booleans one of the examples has a broader brain:

const coercer = z.string().nullish().transform((s) => !!s && s !== "false" && s !== "0");

expect(coercer.parse("false")).toBe(false);
expect(coercer.parse("true")).toBe(true);
expect(coercer.parse("asdf")).toBe(true);
expect(coercer.parse("1")).toBe(true);
expect(coercer.parse("0")).toBe(false);
expect(coercer.parse(undefined)).toBe(false);
expect(coercer.parse(null)).toBe(false);

Edit: url typo fix

markomitranic avatar Jan 10 '24 10:01 markomitranic