protovalidate-python icon indicating copy to clipboard operation
protovalidate-python copied to clipboard

Consider using a RE2 compliant CEL evaluator

Open ddn0 opened this issue 1 year ago • 1 comments

Feature description:

celpy which protovalidate uses does not support RE2-style regular expressions, which is required according to the CEL specification. Consider using a CEL evaluator that does support RE2-style regular expressions.

Problem it solves or use case:

protovalidate implementations should behave similarly across languages but for rules relying on pattern matching, protovalidate-python will consider common patterns like ^foo$ to match a string like foo\n, while other protovalidate implementations would correctly reject it.

The re2 specification indicates that $ should only match end of text unless multiline mode is enabled, and multiline mode is not the default:

$ at end of text (like \z not \Z) or line (m=true)

Additionally, some protovalidate rules are implemented in terms of patterns, e.g., (buf.validate.field).string.uuid which is implemented as the CEL "this == \'\' || this.matches(\'^[0-9a-fA-F]{8}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{4}-[0-9a-fA-F]{12}$\')" which means it can match a string of 37 characters (the normal 36 plus newline).

Examples or references:

echo '"\n"' | python -m celpy 'string(this).matches("^$")'
# Outputs true

which makes sense because all celpy does is call re.search and

import re
assert re.search("^$", "\n") is not None

The cel-go REPL (correctly) does not match this pattern:

$ git clone https://github.com/google/cel-go
$ cd cel-go/repl/main
$ go run .
cel-repl> %let this = '\n'
cel-repl> this.matches('^$')
false : bool

ddn0 avatar Mar 26 '24 15:03 ddn0

@ddn0 we originally planned to do this in #299, but reverted the re2 changes. The google-re2 package has no built wheel for Python 3.13 (issue) and re2 offers no wheels at all. Since building from source would require a C++ toolchain, we don't want to push this burden onto those using the latest version of Python.

Ideally, it would be nice for google-re2 to publish a 3.13 wheel and for your PR in cel-python to get merged, so let's keep this issue open for tracking.

smaye81 avatar May 07 '25 15:05 smaye81

We have solved this in the latest release as well as we're going to be able to (unless we some day get a built wheel for google-re2 for Python 3.13); see the release notes there.

chrispine avatar Jun 23 '25 20:06 chrispine

Thanks @smaye81 , @chrispine for the work and the reasonable workaround!

ddn0 avatar Jun 23 '25 21:06 ddn0