extract engine to own library to facilitate different DSLs?

[jasonkuhrt]

Consider this library https://github.com/flatiron/revalidator

It allows arbitrary validation like:

r.validate({a:5},{properties: {a: { type: 'string' } } } )
{ valid: false,
  errors:
   [ { attribute: 'type',
       property: 'a',
       expected: 'string',
       actual: 'number',
       message: 'must be of string type' } ] }

It seems that the engine behind matches could be extracted to support schema validation, no?

As it stands I would have to include a revalidator to validate resources in my API and then matches for my functions. The seeming duplication is needlessly confusing. While ultimately each might expose a different DSL why couldn't both be powered by the same engine?

As it stands it appears that the matches DSL and engine are strongly coupled?

[natefaubion]

There are two main components to matches: The DSL, and the dispatcher. Which one do you consider to be the "engine"?

The DSL and dispatcher aren't really coupled, as you could use the dispatcher without the DSL. The DSL component has two parts: the parser and the compiler. It just does the typical DSL -> parser -> AST -> compiler -> JS dance. The result being a function that can be fed to the dispatcher. So are you talking about decoupling the parser and compiler? I would say they are already decoupled by using the AST. You can feed a premade AST to matches.compile and get a validation function out. If you wanted, you could write a simple shim that turned a js object (rather than a DSL string) into the matches AST, and get a "validation" function back.

The main problem with using the matches output for validation is that it currently doesn't return any information on why a validation failed. It's designed for dispatch, so it just returns false on match failure as quickly as possible. We would have to augment the compiler to return a MatchError object or something that contained information on why a match failed.

If you just wanted an easier way to feed info to the compiler, there could be another step that turns the AST into some sort of IR tree that's a little easier for human digestion and doesn't contain any of the syntactic info of the DSL.

Edit: That was a bit rambling and didn't really answer your question about factoring it out into its own lib. Yes, it could be done with the above caveats worked out. I would imagine three libs, the code generator, the DSL parser, and a dispatcher.

[jasonkuhrt]

@natefaubion Thanks for the quick reply, and clarifying what I am trying to get at.

The main problem with using the matches output for validation is that it currently doesn't return any information on why a validation failed.

Indeed this would be a primary obstacle for providing validation error feedback for humans. It sounds like you have a much better grasp of the implementation requirements/implications of this feature request.

The end result might be a function that matches any pattern against any value and returns the reasons why it failed. The reasons would undoubtedly have default explanations that formulate like "Name expected to be a String but was actually a number (45)", or more extravagant custom instructions for i.e. "PhoneNumber expected to contain 9 digits but was actually 6 (845 342)"

It seems to me this feature request is to expose low-level and expandable matching to build a validation library on top of.

Thoughts?

[jasonkuhrt]

It might be that pattern matching and validation are each their own library built on top of a much lower-level tool?

[natefaubion]

The lowest level you can get is the code generation. Unlike the flatiron validator, which I assume inspects the schema object each time its called, matches has to compile a validation function. Function dispatch has to be as fast as possible, and the only way to do that is through code generation. So the core would be the code generator, that takes some sort of AST/IR tree, and returns a compiled validation function that inspects the values passed to it, returning MatchErrors with details (as opposed to false). You could take that core, and build a small veneer on top that exposes a "validation" API. The string DSL could then be implemented on top of that, feeding into the code generator. The dispatcher would then be separate, deferring to the DSL when given a string.

So yes, some neat ideas there. If you have any interest in contributing, let me know.

[natefaubion]

Honestly, the AST is about as simple as it can get. I think there are a few references to 'pattern' in the compiler, and it may not strictly be necessary to hold on to that for the compiler. It's mainly for caching purposes.

>>> m.parse('{bob: [...Number]}, x@String')
{ pattern: '{"bob":[...Number]},x@String',
  type: 'argumentList',
  children:
   [ { pattern: '{"bob":[...Number]}',
       type: 'object',
       children:
        [ { pattern: '"bob":[...Number]',
            type: 'keyValue',
            value: '"bob"',
            children:
             [ { pattern: '[...Number]',
                 type: 'array',
                 children:
                  [ { pattern: '...Number',
                      type: 'rest',
                      children: [ { pattern: 'Number', type: 'class', value: 'Number' } ] } ] } ] } ] },
     { pattern: 'x@String',
       type: 'binder',
       value: 'x',
       children: [ { pattern: 'String', type: 'class', value: 'String' } ] } ] }

If we provided MatchErrors, it would be pretty trivial to write a validation lib that generated a subset of the AST based on a schema and fed it to the compiler.

[jasonkuhrt]

Hey, I would like to help out but won't have time for some weeks/months. I will do my best to contribute down the road if the opportunity still exists.