atc-router icon indicating copy to clipboard operation
atc-router copied to clipboard

perf(parser): cache regex predicates with rc using router attribute

Open nowNick opened this issue 5 months ago • 2 comments

Description

This PR is one out of 3 proposed approaches how to optimize memory consumption for specific edge case with ATC Router. The scenario that is being considered is a Router that's defined with the same regular expression in different predicates. Currently Router does not have any ability to remember the regex passed resulting in a lot of copies of the same regex.

Approach in this PR

This PR proposes adding a special attribute to Router struct called regex_cache. It is passed down to the parser and allows the parser to either retrieve the Value::Regex from cache or create a new one and store it there. This approach does not use any singleton pattern. The upside of this solution is easy to track state - no global state. The downside is the requirement to change a lot of functions to "drill" down the regex_cache property to the place where it needs to be used.

Benchmarks

The benchmarking method was to use the commit in this PR: https://github.com/Kong/atc-router/pull/253 on top of each of these PRs. The memory benchmark was done using dhat crate and performance was measured with criterion crate.

Memory consumption:

Baseline (main) Router attribute (this) Thread local (#250) Memoize (#252)
Total (human-readable) 4.54 GB 🔴 242.5 MB 🏆 242.5 MB 🏆 299.6 MB
Total (exact) 4,544,275,926 bytes in 6,762,329 blocks 242,508,176 bytes in 1,922,359 blocks 242,508,176 bytes in 1,922,359 blocks 299,608,085 bytes in 1,962,821 blocks
At t-gmax 194,867,606 bytes in 382,216 blocks 4,713,936 bytes in 41,863 blocks 4,713,936 bytes in 41,863 block 60,538,606 bytes in 72,258 blocks
At t-end 0 bytes in 0 blocks 0 bytes in 0 blocks 108,034 bytes in 168 blocks 35,671,000 bytes in 42 blocks

Performance:

Baseline (main) Router attribute (this) Thread local (#250) Memoize (#252)
660.68 ms 🔴 78.834 ms :trophy: 80.695 ms 86.169 ms

Other PRs Links:

  • (this) https://github.com/Kong/atc-router/pull/251
  • https://github.com/Kong/atc-router/pull/250
  • https://github.com/Kong/atc-router/pull/252

Issue reference:

KAG-3182

nowNick avatar Sep 19 '24 12:09 nowNick