[Feature Request] Allow preprocess even if schema validation fails

[lonewarrior556]

our code has a very loose parse

const parseBasic = (data: any) => {
  const schema = z
    .object({
      name: z.string(),
      letter: z.preprocess(v => (v as string)?.toUpperCase(), z.enum(['A', 'B'])),
    })
    .describe('Basic');

  try {
    return schema.parse(data);
  } catch (error: any) {
    console.warn(schema._def.description + 'SchemaError', {
      ...error.issues,
    });

    return data;
  }
};

parseBasic({ name: 'Jay', letter: 'a' });
// { name: 'Jay', letter: 'A' }

We love that it adds types to our json, and if the json schema is wrong it simply logs a warning.

But there is a few cases were we would like to preprocess the data prior to parsing incase the parsing fails, we would still like to have the preprocess. Unfortunately there is no way to currently accomplish this.

const parseBasic = (data: any) => {
  const schema = z
    .object({
      name: z.string(),
      letter: z.preprocess(v => (v as string)?.toUpperCase(), z.enum(['A', 'B'])),
      fieldThatDefDoesNotExisit: z.string(),
    })
    .describe('Basic');

  
  try {
    data = schema.preProcess(data);  // this does not currently exist
    return schema.parse(data);
  } catch (error: any) {
    console.warn(schema._def.description + 'SchemaError', {
      ...error.issues,
    });

    return data;
  }
};

parseBasic({ name: 'Jay', letter: 'a' });
// *logs warning*
// { name: 'Jay', letter: 'a' } <= ideally still would be  {name: 'Jay', letter: 'A' }'

[JacobWeisenburger]

Is this what you are looking for?

const preSchema = z.object( {
    letter: z.string().toUpperCase(),
} )

const schema = preSchema.passthrough().pipe(
    z.object( {
        name: z.string(),
        letter: z.enum( [ 'A', 'B' ] ),
    } )
)

console.log( preSchema.parse( { name: 'Jay', letter: 'a' } ) ) // { letter: 'A' }
console.log( schema.parse( { name: 'Jay', letter: 'a' } ) ) // { name: 'Jay', letter: 'A' }

If you found my answer satisfactory, please consider supporting me. Even a small amount is greatly appreciated. Thanks friend! 🙏 https://github.com/sponsors/JacobWeisenburger

[lonewarrior556]

Hi @JacobWeisenburger thanks for the quick reply, Alas no, I was just using fieldThatDefDoesNotExisit to trigger a schema error,

The idea is that even though the Schema DOES have a failure we would still like to preprocess. Here is a cleaner example.

onst schema = z.object({
  stringKey: z.preprocess(v => (typeof v === 'string' ? v.toUpperCase() : v), z.enum(['A', 'B'])),
  numberKey: z.preprocess(v => (typeof v === 'number' ? Math.floor(v) : null), z.number()),
});

const data1 = { stringKey: 'a', numberKey: 10.2 };
const data2 = { stringKey: 'notA', numberKey: 10.2 };
const data3 = { stringKey: 'a', numberKey: 'string' };

// currently
schema.parse(data1); // success { stringKey: 'A', numberKey: 10 }
schema.parse(data2); // fail { stringKey: 'notA', numberKey: 10.2 }
schema.parse(data3); // fail { stringKey: 'a', numberKey: 'string' }

// idealy we still want to prePreccess even if parse will fail
schema.preProcess(data1); // { stringKey: 'A', numberKey: 10 }
schema.preProcess(data2); // { stringKey: 'NOTA', numberKey: 10 }
schema.preProcess(data3); // { stringKey: 'A', numberKey: null }

In reality the json error won't be this egregious, we use the parse function mostly to just know that the schema and data are a bit off to fix at a later date.

We still end up using the data, and it would help us narrow down what is a possible if we know that preprocessing DEFINITLY took place.

I think this is currently not possible looking at the documentation.

[JacobWeisenburger]

Is this what you are looking for?

const data1 = { stringKey: 'a', numberKey: 10.2 }
const data2 = { stringKey: 'notA', numberKey: 10.2 }
const data3 = { stringKey: 'a', numberKey: 'string' }

const preSchema = z.object( {
    stringKey: z.union( [
        z.string().toUpperCase(),
        z.any(),
    ] ),
    numberKey: z.union( [
        z.number().transform( v => Math.floor( v ) ),
        z.any().transform( () => null ),
    ] ),
} )

console.log( preSchema.parse( data1 ) ) // { stringKey: 'A', numberKey: 10 }
console.log( preSchema.parse( data2 ) ) // { stringKey: 'NOTA', numberKey: 10 }
console.log( preSchema.parse( data3 ) ) // { stringKey: 'A', numberKey: null }

const schema = preSchema.passthrough().pipe(
    z.object( {
        stringKey: z.enum( [ 'A', 'B' ] ),
        numberKey: z.number(),
    } )
)

console.log( schema.parse( data1 ) ) // { stringKey: 'A', numberKey: 10 }
console.log( schema.safeParse( data2 ).success ) // false
console.log( schema.safeParse( data3 ).success ) // false

If you found my answer satisfactory, please consider supporting me. Even a small amount is greatly appreciated. Thanks friend! 🙏 https://github.com/sponsors/JacobWeisenburger

[lonewarrior556]

It would solve my example but not what I am asking, The idea is, "The data fails to parse but it can still undergo the preprocess"

[JacobWeisenburger]

I really don't understand what you are asking for then, because I feel I have solved the problems you have presented. Please explain a little more what I'm not solving.

[lonewarrior556]

I want the schema to be strict and throw when the data is unexpected, I can catch this and log,

I would also like to have a preprocessor in place that formats the data without needing it to strictly parse.

I was hoping to use the same library, and make use of the "preprocess" that is built into the schema.

[JacobWeisenburger]

I was hoping to use the same library, and make use of the "preprocess" that is built into the schema.

To the best of my knowledge this is the part that can't be solved currently. I can label this issue as a new feature request, but realistically, I highly doubt it will ever be added. But who knows, I could be wrong.

[colinhacks]

It's true that the "ask" here is not well-defined.

It's not clear when Zod should continue execution vs. deeming the input "un-preprocessable". What should be returned if you pass in an array instead of an object into .preProcess? What if there's no letter key in the object? What if your preprocess function throws an error (as it would if the letter value wasn't a string). What if the ZodPreprocess is the endpoint of a ZodPipeline? There are a lot of edge cases in any possible implementation of .preProcess and I don't see a clean way to determine/explain what a .preProcess() method would actually do. Zod is currently all-or-nothing because that's the only way for it's behavior to be consistent and predictable.

You should think about what you really are trying to achieve here. There are ways to make Zod's parsing more customizable or forgiving, including those suggested by Jacob. A method like .preProcess that happily returns garbage shouldn't be useful to you. You should define a schema that carves out exceptions for things like extraneous fields, etc, then use standard .parse.