[VPP-1972] Regression in NAT44ed performance

[vvalderrv]

Description

In CSIT report for 2101, comparison table [0] shows many NAT44ed test have worse performance compared to 2009 (generally around 5-10 percent). Mainly CPS tests, which exercise the slow path.

Bisection identified this [1] change as the cause.

Running per-patch job on that change shows a regression ([2], -5%) also in a new throughput test, which exercises the fast path.

As the two test types use stateful traffic generator mode, their results show greater variance. So I also run a test ([3], -9%) with stateless traffic (in2out direction only, fast path contrary to what the comment claims).

The change [1] claims to be an improvement, with the goal of simplifying the NAT processing logic, so maybe some regression could be expected. But commit message mentions a speed up, although I am not sure which tests would show such a speed up.

Overall, I deem it likely the change has introduced a processing inefficiency somewhere, and it could get fixed. If the regression is expected for legitimate reasons, they should be documented somewhere.

[0] https://docs.fd.io/csit/rls2101/report/_static/vpp/performance-changes-2n-clx-xxv710-2t1c-pdr.txt

[1] https://gerrit.fd.io/r/c/vpp/+/29983

[2] https://gerrit.fd.io/r/c/vpp/+/29983/29#message-27c964d4fbd1baa6645e80212c659997b1dea838

[3] https://gerrit.fd.io/r/c/vpp/+/29983/29#message-85334390efa669a5d21e05d6accc9f1b70d43fac

Assignee

Unassigned

Reporter

Vratko Polak

Comments

• vrpolak (Tue, 10 Aug 2021 12:07:47 +0000): > commit message mentions a speed up, although I am not sure which tests would show such a speed up.

On second read, the speedup probably refers to hairpinning only (not tested in CSIT).

In CSIT report for 2101, comparison table [0]

The change was merged before 2101 release, but only to master branch, when stable/2101 branch was already created. So the 2101 regression was caused by something else, this is a 2106 regression [4]. It affects mainly UDP CPS tests at small scales; for other tests cases, the difference in performance is not big (relative to standard deviation of the results).

[4] https://docs.fd.io/csit/rls2106/report/_static/vpp/performance-changes-2n-clx-xxv710-2t1c-pdr.txt

Original issue: https://jira.fd.io/browse/VPP-1972