core
core copied to clipboard
opnsense
Enable C-states to reduce power consumption and heat generation
Background/Current situation:
FreeBSD has the possibility to enable C-states of the CPU to reduce power consumption when the CPU is idle. Currently (opnsense 22.1.6) only C1 is enabled in the system, which can be verified on CLI by running the command # sysctl dev.cpu |grep cx
Here is the ouput of a J3160. All 4 cores currently only have C1 enabled while the CPU supports C1, C2 and C3:
root@OPNsense:~ # sysctl dev.cpu |grep cx dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.3.cx_usage_counters: 304429829 0 0 dev.cpu.3.cx_usage: 100.00% 0.00% 0.00% last 312us dev.cpu.3.cx_lowest: C1 dev.cpu.3.cx_supported: C1/1/1 C2/2/500 C3/3/1000 dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.2.cx_usage_counters: 352002831 0 0 dev.cpu.2.cx_usage: 100.00% 0.00% 0.00% last 47us dev.cpu.2.cx_lowest: C1 dev.cpu.2.cx_supported: C1/1/1 C2/2/500 C3/3/1000 dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.1.cx_usage_counters: 288856368 0 0 dev.cpu.1.cx_usage: 100.00% 0.00% 0.00% last 305us dev.cpu.1.cx_lowest: C1 dev.cpu.1.cx_supported: C1/1/1 C2/2/500 C3/3/1000 dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.0.cx_usage_counters: 268697840 0 0 dev.cpu.0.cx_usage: 100.00% 0.00% 0.00% last 529us dev.cpu.0.cx_lowest: C1 dev.cpu.0.cx_supported: C1/1/1 C2/2/500 C3/3/1000
By running # sysctl dev.cpu.0.cx_lowest=C3 I can e.g. enable the support of C3 on core 0 of my cpu. After setting c3 on all cores, the output on my sys looks the following:
root@OPNsense:~ # sysctl dev.cpu |grep cx dev.cpu.3.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.3.cx_usage_counters: 83944495 14117757 30415053 dev.cpu.3.cx_usage: 65.33% 10.98% 23.67% last 282us dev.cpu.3.cx_lowest: C3 dev.cpu.3.cx_supported: C1/1/1 C2/2/500 C3/3/1000 dev.cpu.2.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.2.cx_usage_counters: 66185450 18733810 44416761 dev.cpu.2.cx_usage: 51.17% 14.48% 34.34% last 2024us dev.cpu.2.cx_lowest: C3 dev.cpu.2.cx_supported: C1/1/1 C2/2/500 C3/3/1000 dev.cpu.1.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.1.cx_usage_counters: 80708888 13280545 29189120 dev.cpu.1.cx_usage: 65.52% 10.78% 23.69% last 102us dev.cpu.1.cx_lowest: C3 dev.cpu.1.cx_supported: C1/1/1 C2/2/500 C3/3/1000 dev.cpu.0.cx_method: C1/mwait/hwc C2/mwait/hwc C3/mwait/hwc dev.cpu.0.cx_usage_counters: 53052112 11431991 29129079 dev.cpu.0.cx_usage: 56.67% 12.21% 31.11% last 17268us dev.cpu.0.cx_lowest: C3 dev.cpu.0.cx_supported: C1/1/1 C2/2/500 C3/3/1000 root@OPNsense:~ #
Idea/enhancement request Add an entry in the config settings, potentially near powerd and the CPU governor to set/enable additional C-states of the CPU cores ,so the machine consumes less energy during idle time, which also should lower the generated heat.
I haven't had time yet to plug in a power meter and compare the power consumption, so I can't provide any numbers about potential power savings, but I have run several stress tests on my system and wasn't able to generate any network problems/ system issues.
Thank you for creating an issue. Since the ticket doesn't seem to be using one of our templates, we're marking this issue as low priority until further notice.
For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
The easiest option to gain traction is to close this ticket and open a new one using one of our templates.
dev.cpu.0.cx_lowest with value C2 (or lower) can be added to System->Settings->Tunables (repeat for all cores).
Making it the default would be very worthwhile, potentially saving power across many OPNsense installs (and even allowing more turbo headroom for increased performance).
dev.cpu.0.cx_lowest with value C2 (or lower) can be added to System->Settings->Tunables (repeat for all cores).
You just need to change hw.acpi.cpu.cx_lowest, this value will then be used for all CPU cores.
This issue has been automatically timed-out (after 180 days of inactivity).
For more information about the policies for this repository, please read https://github.com/opnsense/core/blob/master/CONTRIBUTING.md for further details.
If someone wants to step up and work on this issue, just let us know, so we can reopen the issue and assign an owner to it.
When fiddling with C-states, beware of the "exit latency" on the lower C-states. Especially C6/C7, but principally anything higher than C1E. Can increase the packet forwarding latency by a couple hundred microseconds up to units of milliseconds. This probably won't harm web browsing or e-mail, but can hurt real bad if you have latency-sensitive traffic, such as a suboptimal database app running a series of SQL request+response or "cursor operations" say between a DMZ and some inner SQL back-end subnet... Same thing on the servers involved, BTW.
In Linux on BayTrail ATOM (and maybe ancestors), C-states higher than C1E were known to cause freezes (at random, when the GPU part was supposed to wake up from the C-state). That's the primary reason for intel_idle.max_cstate=1 at the Linux kernel command line.
Overall I'd suggest that you avoid C-states deeper than C1E in networking equipment. On a quad-core Haswell, with a TDP of 65W that's mostly idle, banning C-states deeper than C1E has increased consumption of the CPU from 4 to 8 Watt. And, to the users running a thick client DB app on that machine, the difference was like night and day. Suddenly they didn't have to wait 15 seconds to open a complicated DB form - it would come up in 1.5s.
