axosyslog icon indicating copy to clipboard operation
axosyslog copied to clipboard

filterx/filterx-scope: write-protected FilterXScope

Open bazsi opened this issue 5 months ago • 4 comments

By making FilterXScope initially write protected and only making it writable at the first actual write, we can delay cloning of variables for read-only access.

For instance, given foo is a dict:

foo = {"foo":{"foo2":"foovalue"}};

And then in a subsequent filterx block we do this:

foo.foo.foo2

Then the following happens without this patch:

  • "foo" is fetched from the parent scope
  • filterx_scope_clone() is invoked, which will create another ref
  • we continue evaluation

This means that unmatched branches will cause clones. This branch changes this by:

  • making the scope initially write protected
  • making it writable at strategic points
  • as long as it is read-only, mutable variables are shared between the current and the previous FilterXScope, no cloning is performed

We mark the scope as writable at the following points:

  • set-subscript, unset, assign, set-attr, update_repr

bazsi avatar Jul 17 '25 07:07 bazsi

I'd appreciate a review by @MrAnno

bazsi avatar Jul 17 '25 19:07 bazsi

Here's the result of my perf test with and without the patch:

bazsi@bzorp3:~/src/syslog-ng/syslog-ng/dbld/install$ time for i in seq 1 200; do cat pan_dump.txt ; done | nc -q0 localhost 2000

real 0m20.895s user 0m0.105s sys 0m0.730s bazsi@bzorp3:~/src/syslog-ng/syslog-ng/dbld/install$ time for i in seq 1 200; do cat pan_dump.txt ; done | nc -q0 localhost 2000

real 0m19.146s user 0m0.095s sys 0m0.768s

The config has 48 non-matching branches and 1 matching one (the last).

bazsi avatar Sep 26 '25 11:09 bazsi

I tried to address the comments here, some of which I found impractical. Let me know what you think.

bazsi avatar Sep 26 '25 11:09 bazsi

We talked about it privately and the recursive mutates_scope check is needed in order to make it work correctly.

real 0m19.146s

Is this reproducible in reverse order? (I experienced deviations at this scale on my laptop while measuring FilterX performance)

MrAnno avatar Sep 29 '25 09:09 MrAnno