Feature request: Plans for alternative parser implementations

[WtfJoke]

Use case

We currently heavily use zod in our projects. The new Parser (Zod) utility could help us (since we validated most of them manually). Since we have use a lot of single purpose lambdas, we try to keep our lambda bundle size low. We noticed that zod takes a good part of that, because their api (as great as it is :D) its not well suited for tree shakability (see https://github.com/colinhacks/zod/issues/2596). We made some experiments in the past with alternatives like valibot.

While I can understand why you choose zod (I would have too, in your situation :D), are there any plans to support other Schema Validation Libraries like valibot in this case? Im curious about your plans :) It feels like the Parser Utility could support multiple schema libraries like the Parameters Utility supports Secrets-Manager and Parameter Sotre.

I am aware of, that you care about bundle size as well, so it seems like a good fit :) I'm also aware that valibot hasnt reached 1.0 and has most likely a breaking api change (see https://github.com/fabian-hiller/valibot/pull/502)

P.S: If you think that idea is better suited in a discussion, I can close this issue.

Solution/User Experience

I just adapted the example of the readme (it probably doesn't compile, just to get the idea)

import type { Context } from 'aws-lambda';
import { isoTimestamp, object, parse, string, type Output } from "valibot";
import middy from '@middy/core';

const TimestampSchema = object({
  timestamp: string([isoTimestamp()]),
});

type Timestamp = Output<typeof TimestampSchema>;

const lambdaHandler = async (
  event: Timestamp,
  _context: Context
): Promise<void> => {
    console.log('Processing event', {event});
};

export const handler = middy(lambdaHandler).use(
  parser({ schema: TimestampSchema })
);

Alternative solutions

Alternative parsers:
- https://github.com/hapijs/joi
- https://github.com/ajv-validator/ajv

Acknowledgment

• [X] This feature request meets Powertools for AWS Lambda (TypeScript) Tenets
• [ ] Should this be considered in other Powertools for AWS Lambda languages? i.e. Python, Java, and .NET

Future readers

Please react with 👍 and your use case to help us understand customer demand.

[boring-cyborg[bot]]

Thanks for opening your first issue here! We'll come back to you as soon as we can. In the meantime, check out the #typescript channel on our Powertools for AWS Lambda Discord: Invite link

[dreamorosi]

Hi @WtfJoke thank you for taking the time to open an issue and for bringing up this topic.

We have considered the idea during development and like you said, the utility lends itself to a model where you could switch the underlying library. However to be frugal with time and pragmatic with efforts, we decided to go with Zod first.

For this first version we settled on Zod mainly because we thought it'd help a wider audience both as a reflection of its larger user base (8.7M downloads/week vs 142K downloads/week for valibot) and the time/effort investment that customers might have already put in creating schemas for it.

As secondary factors, we also took in account the fact that - like you mentioned - valibot is still not stable, and other AWS projects like Amplify have also adopted Zod.

Overall I think that if valibot becomes stable and Powertools customers ask for it, we could consider adding this capability in a future major version. Since we maintain a collection of schemas and envelopes for AWS events, this would require us doubling the maintenance for each provider we add, however as long as the Parser utility gains traction and there's significant customer demand, I don't think it's out of question.

I think we should leave the issue open so that people can express their interest.

[WtfJoke]

Thanks for the insights! Totally understandable

[dreamorosi]

As a side note, that doesn't in any way invalidate the ask, I also wanted to surface that Zod appears to be working on a v4 that should improve with tree shaking & bundle size. We are keeping an eye on the development and will consider adopting once it's out.

[fabian-hiller]

I am the creator of Valibot and happy to help and answer questions if you consider supporting Valibot. I expect to release a stable version within the next 2 or 3 months.

[dreamorosi]

Circling back to this after a few months to provide an update.

The utility went GA (Generally Available) in the last release and has seen decent usage so far. Our position on this topic hasn't changed and we are still open to consider alternative libraries provided there's demand and a stable alternative.

We'll revisit this again in ~3 months to see if either has changed. In the meantime perhaps the Parser utility will have gained additional usage.

[fabian-hiller]

Valibot should reach v1 RC status soon.

[dreamorosi]

There's a new standardization effort that is getting some traction called standard-schema that seems to offer a shared interface for Zod, Valibot, and others.

In their own words:

A consortium of schema library authors have collaborated to craft a standard interface for schema libraries to benefit the entire JavaScript ecosystem. Standard Schema provides third-party libraries a uniform integration to automatically support multiple schema libraries at once, without adding a single runtime dependency. This simplifies implementation, prevents vendor lock-in, and enables innovation, especially for smaller schema libraries with new ideas.

We should look into this and understand how an integration would look like, as well as what changes would be needed on our side to support this.

If it works as it says it does, we might be able to support any validation & parsing library that adheres to this standard schema out of the box.

cc @am29d

[fabian-hiller]

Reach out if you have any questions about Standard Schema. I am happy to help!

[am29d]

This looks interesting, will take a closer look!

[dreamorosi]

Hey @fabian-hiller - thank you for the offer.

I've been looking at Standard Schema and I think it's the way to go. I have also seen it being used successfully in some codebases like OpenAuth from the SST team and for users it's very seamless.

Over the coming weeks I'm gonna spend some time researching further how we can implement this here, I'm mainly interested in understanding which features are considered interoperable and which are not. I'm quite clear on base schemas being portable but I am still unsure about concepts like .pipe(), .transform() and general error shapes from Zod.

These are all features we use quite a bit in the codebase so I'd need to understand how to make them generic. Plus I also need to figure out types, but in that sense I have some examples like I mentioned above so I'm not too concerned.

If the offer is still valid, I might tag you here when I'm done with the research in case I have questions. Having your input would be extremely valuable. I really want to support Valibot now that it's getting closer and closer to v1 stable.

Also, in case you have any examples of the topics I mentioned above, it'd be much appreciated.

[fabian-hiller]

Standard Schema only covers input type, output type, and validation of unknown data. It does not cover any vendor specific features. But there is a .vendor prop in case you need to do special things depending on the schema library used.

[codyfrisch]

@dreamorosi I am fully supportive of Standard Schema - as I too would like to use libraries other than Zod when pertinent. Zod has been great (I use it pretty heavily) but itself, its heavy too and it has had an impact on the project.

[dreamorosi]

Just wanted to inform everyone following the issue that we are going to move forward with Standard Schema support for our parser middleware and TypeScript class method decorator.

This step will allow us to open up support for Zod v3 & v4 in a backward-compatible way.

[powertools-for-aws-oss-automation[bot]]

[!warning] This issue is now closed. Please be mindful that future comments are hard for our team to see. If you need more assistance, please either reopen the issue, or open a new issue referencing this one. If you wish to keep having a conversation with other community members under this issue feel free to do so.