Validation over multiple fields

[tommyJimmy87]

I have a message like this :

message Request {
    repeated NewRequest add = 1;
    repeated RemoveRequest remove = 1;
}

I would like to validate if in the json there's at least one item of the 2 repeated and if in the json the field is called really add or remove (check for example the camel cases).

{
 "add": [{
 }]
}

and not like this for example:

{
 "Add": [{
 }]
}

[htuch]

At some point, we should consider switching to a more expressive language like https://github.com/google/cel-spec.

[moderation]

Or Open Policy Agent's Rego @tsandall

[tsandall]

Thanks for the mention @moderation. From a language standpoint, Rego is up to the task, but since we don't have a C++ implementation (today), it's probably not a great fit. Perhaps once we finish the Wasm compiler, we could revisit this... :thinking:

[rodaine]

Hey! I'm not sure if this has been discussed on here or not, as @htuch mentioned the plan is to include a predicate language that is only evaluated by the plugin (so no issue wrt C++ there @tsandall), the emitted code would be generated from that and would be language implementation specific.

This hasn't quite been prioritized yet, and we haven't shopped around/spec'd what this would look like. Most basic requirement is that the parser has a Go implementation since the plugin is Go. I'd prefer to not roll our own if we can swing it :)

[kyessenov]

I'm working on open sourcing cel-cpp implementation that implements cel-spec in C++. cel-go has also seen some significant progress recently. There's been some prior art on proto contracts, that we could leverage if you're interested. cc @TristonianJones

[glerchundi]

In case you didn’t noticed, there is a cel-cpp implementation already under the google’d umbrella: https://github.com/google/cel-cpp.

On Thu, 4 Apr 2019 at 21:10, Kuat [email protected] wrote:

I'm working on open sourcing cel-cpp implementation that implements cel-spec in C++. cel-go has also seen some significant progress recently. There's been some prior art on proto contracts, that we could leverage if you're interested.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/envoyproxy/protoc-gen-validate/issues/128#issuecomment-480025808, or mute the thread https://github.com/notifications/unsubscribe-auth/ACIPlupThN26iOuLKxxJdAgXlFqzlDSWks5vdk4cgaJpZM4ZWI3Z .

[glerchundi]

Ignore me, you’re one of those 🤦‍♂️

On Fri, 5 Apr 2019 at 07:13, Gorka Lertxundi [email protected] wrote:

In case you didn’t noticed, there is a cel-cpp implementation already under the google’d umbrella: https://github.com/google/cel-cpp.

On Thu, 4 Apr 2019 at 21:10, Kuat [email protected] wrote:

I'm working on open sourcing cel-cpp implementation that implements cel-spec in C++. cel-go has also seen some significant progress recently. There's been some prior art on proto contracts, that we could leverage if you're interested.

— You are receiving this because you are subscribed to this thread. Reply to this email directly, view it on GitHub https://github.com/envoyproxy/protoc-gen-validate/issues/128#issuecomment-480025808, or mute the thread https://github.com/notifications/unsubscribe-auth/ACIPlupThN26iOuLKxxJdAgXlFqzlDSWks5vdk4cgaJpZM4ZWI3Z .

[glerchundi]

As per this comment it seems like you more o less decided to use the go parser instead of cel, is it possible to elaborate on this and the reasoning behind that decision?

Using one or the other, and correct if i'm wrong. the idea would be to create an expression associated to the message (no matter it is go or cel), parse in protoc-gen time to obtain the AST of the expression and then generate code on each language based on this AST, right?

Thanks!

/cc @rodaine

[rodaine]

That about sums it up, yes :)

[glerchundi]

@rodaine and what about the first question regarding to the reasoning behind deciding to use go/parser instead of CEL? I would like to start working on this and wanted to know if I should start with one or the other!

Thanks!

[rodaine]

I recommended the go parser because it's part of the standard library and does not require any dependencies. The generated code should not have a dependency on whichever expression language is used, so there's no real benefit from my perspective to use a 3rd-party one.

[akonradi]

I agree that the generated code should not have a dependency on the expression language, but that's not the only place there's a dependency. Users of PGV are going to have to write expressions in whatever language we choose, and they may not be familiar with Go or CEL. I think it makes sense to adopt a widely used language instead of writing our own DSL, but it would be helpful to see how some illustrative sets of constraints would be expressed in each before making a selection.

[rodaine]

Totally agree :)

[TristonianJones]

CEL was designed to evaluate over JSON, proto2, and proto3 messages. Most languages aren't. Also, CEL shares much of its syntax with Go syntax, but this is also true of CEL and any C-like language.

In my prior experience with codegen from CEL, I've found that it can be challenging to map from proto types to runtime specific equivalents, i.e. Java and unsigned ints. Also, it can be slow to add new functions and operator overloads.

There has been some talk of trying to use CEL within Envoy filters. If this were to happen, I would see that as strong motivation for using CEL in PGV. If you're collecting canonical use cases and comparing syntax, I would be happy to help. Also, if there are specific concerns about CEL tooling, I'm pretty flexible when it comes to restructuring the code for speed, ease of use, and dependency footprint.

/cc @kyessenov

[inigohu]

Could you consider a middle way? For example, add conditionals as AND/OR. This would solve many use cases without having to implement CEL.

[elliotmjackson]

Good news! this is now possible in protovalidate.

syntax = "proto3";

import "google/protobuf/timestamp.proto";
import "buf/validate/validate.proto";

message Transaction {
  google.protobuf.Timestamp purchase_date = 1;
  google.protobuf.Timestamp delivery_date = 2;

  option (buf.validate.message).cel = {
    id: "transaction.delivery_date",
    message: "Delivery date must be after purchase date",
    expression: "this.delivery_date > this.purchase_date"
  };
}