jsonschema-rs icon indicating copy to clipboard operation
jsonschema-rs copied to clipboard

feat: configure patterns regex engine

Open shumkov opened this issue 1 year ago • 4 comments

Fancy Regex supports backtracking which is required for some cases but as a downside is vulnerable to ReDoS attacks. This becomes a decisive factor when an application operates with user-defined schemas. Regex, in turn, doesn't support look-around and backreferences but guarantees linear time matching that mitigates the attack.

This PR enables the configuration of the regex engine for pattern-based keywords: Regex or FancyRegex (by default).

use jsonschema::{CompilationOptions, RegexEngine, RegexOptions};
let mut options = CompilationOptions::default();
// Set Regex as a default engine for pattern keyword
options.with_patterns_regex_engine(RegexEngine::Regex(RegexOptions {
  size_limit: Some(5 * (1 << 20)),
  ..Default::default()
}));

The formats still use Fancy Regex. I didn't find a simple way to keep patterns static and configurable at the same time. Probably, the right approach is to add an option to use fast formats such as ajv-formats but this is out of the scope of this PR.

shumkov avatar May 21 '24 17:05 shumkov

Hello @Stranger6667! Will it be interesting for you guys or it's just my specific use case?

shumkov avatar Jun 13 '24 14:06 shumkov

This looks cool! Sorry for the delay, I’ll check it in detail in the next couple of days and will let you know

Stranger6667 avatar Jun 13 '24 14:06 Stranger6667

Codecov Report

Attention: Patch coverage is 67.05882% with 28 lines in your changes missing coverage. Please review.

Project coverage is 89.69%. Comparing base (8adae12) to head (7b00a24). Report is 347 commits behind head on master.

Files with missing lines Patch % Lines
jsonschema/src/regex.rs 51.16% 21 Missing :warning:
jsonschema/src/compilation/options.rs 63.63% 4 Missing :warning:
jsonschema/src/keywords/pattern.rs 70.00% 3 Missing :warning:
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #487      +/-   ##
==========================================
- Coverage   89.90%   89.69%   -0.21%     
==========================================
  Files          58       59       +1     
  Lines        9942    10007      +65     
==========================================
+ Hits         8938     8976      +38     
- Misses       1004     1031      +27     

:umbrella: View full report in Codecov by Sentry.
:loudspeaker: Have feedback on the report? Share it here.

:rocket: New features to boost your workflow:
  • :snowflake: Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

codecov[bot] avatar Jun 13 '24 19:06 codecov[bot]

CodSpeed Performance Report

Merging #487 will degrade performances by 20.9%

Comparing dashpay:configure_regexp (7b00a24) with master (8adae12)

Summary

⚡ 15 improvements ❌ 10 regressions ✅ 295 untouched benchmarks

:warning: Please fix the performance issues or acknowledge them on CodSpeed.

Benchmarks breakdown

Benchmark master dashpay:configure_regexp Change
fast jsonschema/is_valid/invalid 829.2 ns 945.8 ns -12.33%
jsonpointer[empty] 372.5 ns 429.7 ns -13.32%
additional_items_object 123foo[jsonschema/is_valid/valid] 1.3 µs 1.2 µs +12.55%
all_of 1[jsonschema/is_valid/invalid] 1,004.4 ns 887.8 ns +13.14%
any_of_multiple_types foo[jsonschema/is_valid/valid] 941.1 ns 853.6 ns +10.25%
any_of_multiple_types null[jsonschema/is_valid/invalid] 941.9 ns 854.4 ns +10.24%
contains 1[jsonschema/is_valid/invalid] 643.1 ns 759.7 ns -15.36%
contains 5[jsonschema/is_valid/valid] 613.1 ns 700.6 ns -12.49%
exclusive_maximum 2[jsonschema/is_valid/valid] 331.1 ns 418.6 ns -20.9%
exclusive_maximum 3[jsonschema/is_valid/invalid] 331.1 ns 418.6 ns -20.9%
exclusive_minimum 3[jsonschema/is_valid/invalid] 301.7 ns 360 ns -16.2%
exclusive_minimum 4[jsonschema/is_valid/valid] 301.7 ns 360 ns -16.2%
format_email foo[jsonschema/is_valid/invalid] 614.7 ns 527.2 ns +16.6%
format_email [email protected][jsonschema/is_valid/valid] 624.7 ns 537.2 ns +16.29%
format_iri http//ƒøø.ßår/?∂éœ=πîx#πîüx[jsonschema/is_valid/valid] 23.6 µs 21.3 µs +10.58%
items 123[jsonschema/validate/valid] 4.3 µs 5.2 µs -18.71%
items 12x[jsonschema/validate/invalid] 8.1 µs 9.1 µs -10.33%
max_length aaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaaa[jsonschema/is_valid/invalid] 848.1 ns 760.6 ns +11.5%
max_length foo[jsonschema/is_valid/valid] 665.6 ns 578.1 ns +15.14%
maximum 3[jsonschema/is_valid/valid] 360 ns 301.7 ns +19.34%
... ... ... ... ...

:information_source: Only the first 20 benchmarks are displayed. Go to the app to view all benchmarks.

codspeed-hq[bot] avatar Jun 13 '24 19:06 codspeed-hq[bot]

This feature will be available in 0.30.0, sorry for the delay

Stranger6667 avatar Apr 16 '25 19:04 Stranger6667