FrameworkBenchmarks icon indicating copy to clipboard operation
FrameworkBenchmarks copied to clipboard

The last run the numbers are lower

Open joanhey opened this issue 1 year ago • 5 comments

The run before: image

The last run: image

joanhey avatar Dec 08 '24 23:12 joanhey

Perhaps it's another security mitigation. But it's good to know from where is the problem.

PD: normally appear when we restart the servers.

joanhey avatar Dec 08 '24 23:12 joanhey

also some numbers are not correct! image

Xudong-Huang avatar Dec 09 '24 02:12 Xudong-Huang

The Linux kernel was updated from version 6.5.0-41-generic to 6.8.0-49-generic.

volyrique avatar Dec 10 '24 17:12 volyrique

Perhaps will be good to run lscpu after each new restart.

So we can check for new Vulnerabilities:

Vulnerabilities:
  Itlb multihit:         Not affected
  L1tf:                  Not affected
  Mds:                   Not affected
  Meltdown:              Not affected
  Mmio stale data:       Mitigation; Clear CPU buffers; SMT vulnerable
  Retbleed:              Not affected
  Spec store bypass:     Mitigation; Speculative Store Bypass disabled via prctl and seccomp
  Spectre v1:            Mitigation; usercopy/swapgs barriers and __user pointer sanitization
  Spectre v2:            Mitigation; Enhanced IBRS, IBPB conditional, RSB filling, PBRSB-eIBRS SW sequence
  Srbds:                 Not affected
  Tsx async abort:       Not affected

PD: some platforms loss more than 25% (-1 million requests). We had the same problem sometime ago with the vulnerabilities mitigations.

joanhey avatar Dec 20 '24 10:12 joanhey

After this run with lower numbers, in each new run we have a degradation in performance.

Examples are the first fortune framework (may-minihttp) and plain nginx plaintext.

The diff in numbers:

Framework Run Run (server restart) Runs after
may-minithttp (fortunes) 1,440,495 1,343,512 1,331,796 1,317,465
nginx (plaintext) 5,828,101  4,930,020 4,886,378 4,770,160

With graphs:

may-minitthp nginx
image image

Perhaps it's temporal, but it'll be good to check the next runs!!

joanhey avatar Jan 05 '25 10:01 joanhey