bufbuild
[Feature Request] Typescript Support
Feature description:
Introduce TypeScript support to the protovalidate library for performing runtime validation of Protocol Buffers messages in TypeScript projects. This was a feature request in protovalidates predecessor bufbuild/protoc-gen-validate#71
Problem it solves or use case:
The absence of TypeScript support in the protovalidate library hinders developers who work with Protocol Buffers and TypeScript. By adding TypeScript support, developers would be able to validate Protocol Buffers messages at runtime within their TypeScript applications, enhancing data integrity and reducing potential bugs related to data validation.
Proposed implementation or solution:
The proposed solution involves extending the existing protovalidate library to include TypeScript bindings and support. This includes defining TypeScript type definitions that correspond to Protocol Buffers message structures, and implementing runtime validation functions that work seamlessly with TypeScript types.
Contribution: I am willing to contribute to this feature request by actively participating in discussions, assisting in defining TypeScript type definitions, and collaborating on the implementation of runtime validation functions for TypeScript. If necessary, I can also assist in writing tests and documentation to ensure the feature is thoroughly covered and well-documented.
Examples or references:
While protovalidate doesn't currently have TypeScript support, similar libraries like protobufjs and protobuf-ts provide inspiration for how TypeScript bindings and runtime validation can be integrated for Protocol Buffers messages in a type-safe manner.
Additional context:
Enabling TypeScript support in the protovalidate library would bridge the gap for developers who use Protocol Buffers in TypeScript projects, allowing them to perform runtime validation without relying on separate code generation processes. This aligns with the trend of utilizing TypeScript's strong typing and safety features in modern software development.
Good to hear that, maybe i can also contribute. It seems like zod is popular these days, you would instantly also have validation up to the frontend: https://zod.dev/
@elliotmjackson Hi there! Is there already something available to try out?
Working on this is on our roadmap, but progress is blocked on a solution to https://github.com/bufbuild/protobuf-es/issues/397 (which is also on our roadmap). We will share any updates when we have them.
Hi there! Having typescript support for protovalidate would be fantastic.
I noticed in the linked blocker above that @protobuf-ts correctly includes options in the generated typescript. I was able to successfully test this out locally, such that my generated Typescript code contained custom proto options I defined using protovalidate syntax.
Would you folks be open to adding library functions that don't depend on protobuf-es while that blocker exists? For example, instead of providing a function public ValidationResult validate(Message msg) within the library, you could instead expect that:
- Users of your library would define their validation in their
.protofiles (no change here). - Users of your library would be responsible for generating
FieldConstraintsandMessageConstraintsand such. (As a user, I would rely onprotobuf-tsto help me do this with its helper functions likereadFieldOption.) - Your Typescript protovalidate library would provide methods like
public ValidationResult validate(fieldValue Value, fieldConstraints FieldConstraints).
I believe with this structure, your library wouldn't have any need to inspect what options were defined on a given proto message. Instead, clients would be responsible for doing this themselves, and your protovalidate Typescript library would simply provide the validation component. Long-term it would obviously be wonderful to address https://github.com/bufbuild/protobuf-es/issues/397 and have a simple validate method available that could parse custom proto options, similar to what's present in the Python and Golang and Java libraries you folks have developed. But it would definitely be helpful to have a stop-gap solution here that at least provided partial support for users of your protovalidate ecosystem!
Would you folks be open to adding library functions that don't depend on protobuf-es while that blocker exists?
No, we are focusing our attention on improving protobuf-es and are actively working on a solution to the blocking issue.
Would you folks be open to adding library functions that don't depend on protobuf-es while that blocker exists?
No, we are focusing our attention on improving protobuf-es and are actively working on a solution to the blocking issue.
Thanks Nick, that's great to hear! Do you folks have any ETA on when you might have the blocking issue resolved, and when it might be possible to start developing the protovalidate-typescript library? (I'd be happy to help out with the latter if you're looking for contributors!)
Rough guess is that we will be unblocked sometime in Q1. Appreciate the offer to help and will post to the issue if we could use help when the time comes.
Since protovalidate relies heavily on CEL does this mean that a CEL typescript evaluator is being developed as well? And will there be a standalone version of the evaluator? We have the need for a javascript/typescript CEL evaluator in our project but we can't seem to find a robust implementation at the moment.
Yes, supporting Protovalidate in TypeScript entails having a high quality open source CEL interpreter that works in all the environments we want (aka doesn't require Node.js APIs and doesn't require WASM). We are also unaware of any existing open source project that meets these requirements and so have already invested some time building our own implementation. We would open source this CEL implementation at the same time that we would open source support for Protovalidate TypeScript (which currently doesn't have a concrete timeline associated with it).
Is there a clearer timeline on this? We're adding some protobuf-based metadata and being able to add validation would be really helpful. But we do need Typescript support.
We do not have a clearer timeline for this at this point. As we have been working on https://github.com/bufbuild/protobuf-es/issues/397 and adding support for Protobuf Editions, it became apparent that we need to release a v2 of protobuf-es to support both of these things, so the rough timeline I gave earlier is no longer accurate. The good news is there is an alpha pre-release of protobuf-es v2 already, but there is more work to do. Once protobuf-es v2 is done and https://github.com/bufbuild/protobuf-es/issues/397 is closed, work on Protovalidate TypeScript will not be blocked.
Will we need to be on protobuf-es in order to use protovalidate? We currently use protobuf-ts.
Yes, you will need to be using protobuf-es. The runtimes differ to much for us to support both.
@nicksnyder,
looks like a bunch of progress has been made on protobuf-es v2, including a release of not just v2.0.0 but also v2.1.0. congrats on those huge milestones!
i'm assuming this means that protovalidate-ts is also in flight as it seems the identified blockers have been removed.
is there anything needed by the buf from the community on testing this or any other help?
In addition to releasing protobuf-es v2 we have also needed to follow through with connect-es v2 because that depends on protobuf-es. That body of work is wrapping up and we have started to internally discuss our plans to move Protovalidate TS forward, but don't have anything concrete to share publicly yet. I know there are a lot of people interested in this and we want it too!
If it's helpful at all (even if just for prior art), I'm working on a CEL typescript implementation based on the cel-go package. The parser and checker work. Just need to finish implementing the interpreter. I know a CEL interpreter is a prerequisite for protovalidate and one doesn't currently exist. https://github.com/jafaircl/bearclaw/tree/issue-35/packages/cel
