Value validation (Is there interest in this feature?)

[stevendesu]

After a lot of searching I couldn't find a library or tool that did what I wanted, so I started to build it myself when I realized that 90% of the parts were already in place within typescript-is, and it would be much faster to fork typescript-is and add this feature than to build it all out from scratch. So long as I'm working on this fork, I wanted to see if there was any interest on merging this (or a similar feature) into the main code base.

High-level Overview

I'm looking for a tool that can perform value validation on wild objects (generally the result of calling JSON.parse on API responses). So if a value is supposed to be, say, between 0 and 100, then if we get back a proper type ("number") but the value is invalid ("-1") we'll throw an error.

Furthermore I wanted the ability to handle errors in different ways based on which field was in error. Some fields in our data are necessary in order to properly process and handle information while other fields are optional or contain excess metadata. If one of these options or metadata fields has an error in it, it shouldn't prevent us from processing the rest of the message. To this end I was thinking there were 3 valid ways to handle a message that doesn't pass validation:

• Set the invalid field to null (ignore that field, process the rest of the message)
• Set the whole object to null (scrap this message, continue to process others)
• Throw an error (give up, die)

Finally, I wanted all of this functionality without having to write a lot of additional code or run additional tools. Many schema validators require you to define your types first in TypeScript, then again in a schema file. Some work around this by letting you just define them in a schema file, then the schema file is compiled to a TypeScript definition file using a tool -- but this is an extra step for developers and requires re-running the tool if your schema changes.

My Plan to Accomplish This

I wanted to leverage block comments to pass metadata about types, including validation rules and instructions on how to handle failure. An example would be:

/**
 * @validate fields
 */
export type Animal = {
    /**
     * The name of the animal, if it's a pet or otherwise has been named
     * @regex /[A-Z][a-z]+/
     * @invalid null
     */
    name?: string;

    /**
     * The age of the animal, if known
     * @min 0
     * @max 99
     * @invalid throw
     */
    age?: number
};

The advantage to using comments like this is that the validation rules show up in type hints in most IDEs:

Parsing out the comments is unfortunately not as straight-forward with the TypeScript AST as calling something like node.getJSDoc(), but it's not too difficult once you look through the API:

const leadingCommentRanges = ts.getLeadingCommentRanges(node.getFullText(), 0);
if ( ! leadingCommentRanges) return ts.visitEachChild(node, visitor, context);

const lastCommentRange = leadingCommentRanges[leadingCommentRanges.length - 1];
if (lastCommentRange.kind !== ts.SyntaxKind.MultiLineCommentTrivia) return ts.visitEachChild(node, visitor, context);

const lastComment = node.getFullText().substring(lastCommentRange.pos, lastCommentRange.end);
if ( ! lastComment.startsWith("/**")) return ts.visitEachChild(node, visitor, context);

// At this point we have a JSDoc. Now we're just hunting for validator parameters

const lines = lastComment.split("\n");
lines.forEach(line => {
	// Remove preceding white space and up to 1 preceding asterisk
	const trimmed = line.replace(/^\s*\*?\s*/, "");
	if (trimmed.startsWith("@min")) {
		// Found one
	} else if (trimmed.startsWith("@max")) {
		// Found another
	} else if (trimmed.startsWith("@regex")) {
		// Found another
	} // ...
});

Before I get started on this fork of typescript-is, I just wanted to make a post here to see if there are any plans to add similar functionality, or if there would be any interest in helping to build this out. I was also curious if there would be any interest on merging this into the main code base once I was done.

I can't be sure how long it will take me to finish since I've got some competing priorities for my time. At work we have a less-than-ideal work-around for validating our messages (manually writing out the validation functions) so there's no a lot of support from above to do this during work hours, and at home I've got a toddler. So unless someone is willing to help me with this undertaking I wouldn't expect to have any meaningful work to show for at least a couple of months.

[stevendesu]

Some more specifics for the ideas I planned on implementing:

• Types, interfaces, and classes can have the optional JSDoc-style comment tag @validate specifying how validation should function. It's followed by an enumerated string with one of the values: all, fields, or throw. In the case of all, all fields must validate or the is<Type>() check returns false (we don't process the message). In the case of fields, the fields are processed one at a time as described below. In the case of throw, the is<Type>() check will throw an error instead of returning false. Defaults to all
• Properties can have optional JSDoc-style comment tags with validation rules: @min (numbers only), @max (number only), @length (will count digits for numbers or characters for strings), or @regex (strings only). Other may be added later as I think of them
• Properties can have the optional JSDoc-style comment tag @invalid specifying how to handle invalid fields. It's followed by an enumerated string with one of the values: null, fail, or throw. In the case of null, if the field fails to validate it shall be set to null (assuming the type is nullable -- if not, it will fall back to throw). In the case of fail, if the field fails to validate then the whole type will fail to validate and false will be returned by is<Type>(). In the case of throw, if the field fails to validate then an error is thrown. Defaults to fail

The reason for the throw option is to prevent further processing of messages if one invalid messages spoils the batch, such as is the case with aggregation. The default values should prevent throwing of unexpected errors, however, to conform to the default behaviour of typescript-is.

[samchon]

@stevendesu Some features are supported by my libraries

https://github.com/samchon/typescript-json/#comment-tags