CVE-2017-5715 / Spectre Variant 2 on RHEL 7.5 (Broadwell)

[knweiss]

The following system with Broadwell CPU uses the Red Hat/CentOS 7.5 defaults regarding Spectre mitigation. (FWIW: This is a virtual machine!)

Unfortunately, spectre-meltdown-checker and Red Hat's sysfs status disagree about the mitigation status of Spectre Variant 2:

spectre-meltdown-checker says this:

# spectre-meltdown-checker  --batch nrpe --variant 2
Vulnerable: CVE-2017-5715

Red Hat's sysfs file:

# cat /sys/devices/system/cpu/vulnerabilities/spectre_v2
Mitigation: Full retpoline
# grep . /sys/kernel/debug/x86/*_enabled
/sys/kernel/debug/x86/ibpb_enabled:1
/sys/kernel/debug/x86/ibrs_enabled:0
/sys/kernel/debug/x86/pti_enabled:1
/sys/kernel/debug/x86/retp_enabled:1
/sys/kernel/debug/x86/ssbd_enabled:3

Red Hat's mitigation defaults are described here and the defaults for this pre-Skylake CPU with updated microcode are:

Intel Defaults:
    pti=1 ibrs=0 retp=1 ibpb=1-> fix variant#1 #2 #3 for pre-Skylake cpus 

However, spectre-meltdown-checker (commit dd67fd9) complains about the disabled IBRS:

# spectre-meltdown-checker --variant 2 --explain -v                                                                                                                                                        [10/59836]
Spectre and Meltdown mitigation detection tool v0.39+

Checking for vulnerabilities on current system
Kernel is Linux 3.10.0-862.11.6.el7.x86_64 #1 SMP Tue Aug 14 21:49:04 UTC 2018 x86_64
CPU is Intel(R) Xeon(R) CPU E5-2697A v4 @ 2.60GHz
Will use kernel image /boot//vmlinuz-3.10.0-862.11.6.el7.x86_64
Will use kconfig /boot/config-3.10.0-862.11.6.el7.x86_64
Will use System.map file /proc/kallsyms
Kernel image is Linux version 3.10.0-862.11.6.el7.x86_64 ([email protected]) (gcc version 4.8.5 20150623 (Red Hat 4.8.5-28) (GCC) ) #1 SMP Tue Aug 14 21:49:04 UTC 2018

Hardware check
* Hardware support (CPU microcode) for mitigation techniques
  * Indirect Branch Restricted Speculation (IBRS)
    * SPEC_CTRL MSR is available:  YES 
    * CPU indicates IBRS capability:  YES  (SPEC_CTRL feature bit)
  * Indirect Branch Prediction Barrier (IBPB)
    * PRED_CMD MSR is available:  YES 
    * CPU indicates IBPB capability:  YES  (SPEC_CTRL feature bit)
  * Single Thread Indirect Branch Predictors (STIBP)
    * SPEC_CTRL MSR is available:  YES 
    * CPU indicates STIBP capability:  YES  (Intel STIBP feature bit)
  * Speculative Store Bypass Disable (SSBD)
    * CPU indicates SSBD capability:  YES  (Intel SSBD)
  * L1 data cache invalidation
    * FLUSH_CMD MSR is available:  YES 
  * Enhanced IBRS (IBRS_ALL)
    * CPU indicates ARCH_CAPABILITIES MSR availability:  YES 
    * ARCH_CAPABILITIES MSR advertises IBRS_ALL capability:  NO 
  * CPU explicitly indicates not being vulnerable to Meltdown (RDCL_NO):  NO 
  * CPU explicitly indicates not being vulnerable to Variant 4 (SSB_NO):  NO 
  * Hypervisor indicates host CPU might be vulnerable to RSB underflow (RSBA):  YES 
  * CPU microcode is known to cause stability problems:  NO  (model 0x4f family 0x6 stepping 0x1 ucode 0xb00002e cpuid 0x406f1)
  * CPU microcode is the latest known available version:  YES  (you have version 0xb00002e and latest known version is 0xb00002e)
* CPU vulnerability to the speculative execution attack variants
  * Vulnerable to Variant 1:  YES 
  * Vulnerable to Variant 2:  YES 
  * Vulnerable to Variant 3:  YES 
  * Vulnerable to Variant 3a:  YES 
  * Vulnerable to Variant 4:  YES 
  * Vulnerable to Variant l1tf:  YES 

CVE-2017-5715 [branch target injection] aka 'Spectre Variant 2'
* Mitigated according to the /sys interface:  YES  (Mitigation: Full retpoline)
* Mitigation 1
  * Kernel is compiled with IBRS support:  YES  (/sys/kernel/debug/x86/ibrs_enabled exists)
    * IBRS enabled and active:  NO 
  * Kernel is compiled with IBPB support:  YES  (/sys/kernel/debug/x86/ibpb_enabled exists)
    * IBPB enabled and active:  YES 
* Mitigation 2
  * Kernel has branch predictor hardening (arm):  NO 
  * Kernel compiled with retpoline option:  YES 
    * Kernel compiled with a retpoline-aware compiler:  YES  (kernel reports full retpoline compilation)
    * Retpoline is enabled:  YES 
    * Local gcc is retpoline-aware:  YES 
  * Kernel supports RSB filling:  NO 
> STATUS:  VULNERABLE  (IBRS+IBPB or retpoline+IBPB+RBS filling, is needed to mitigate the vulnerability)

> How to fix: To mitigate this vulnerability, you need either IBRS + IBPB, both requiring hardware
support from your CPU microcode in addition to kernel support, or a kernel compiled with retpoline
and IBPB, with retpoline requiring a retpoline-aware compiler (re-run this script with -v to know
if your version of gcc is retpoline-aware) and IBPB requiring hardware support from your CPU
microcode. You also need a recent-enough kernel that supports RSB filling  if you plan to use
retpoline. For Skylake+ CPUs, the IBRS + IBPB approach is generally preferred as it guarantees
complete protection, and the performance impact is not as high as with older CPUs in comparison
with retpoline. More information about how to enable the missing bits for those two possible
mitigations on your system follow. You only need to take one of the two approaches.

> How to fix: Both your CPU and your kernel have IBRS support, but it is currently disabled.
You may enable it with `echo 1 > /sys/kernel/debug/x86/ibrs_enabled`.

• Are we sure that spectre-metldown-checker is right and Red Hat is wrong?
• Why doesn't the latest Red Hat kernel have RSB filling support?
• It would be great to add an explanation why RSB filling is needed on this pre-Skylake CPU.

Side note: There's a small typo in the VULNERABLE line: "RSB filling", not "RBS filling".

[knweiss]

AFAICS this is the explanation why RSB fill should be enabled unconditionally on all Spectre v2 vulnerable CPUs. Quote:

        * If spectre v2 protection has been enabled, unconditionally fill
        * RSB during a context switch; this protects against two independent
        * issues:
        *
        *      - RSB underflow (and switch to BTB) on Skylake+
        *      - SpectreRSB variant of spectre v2 on X86_BUG_SPECTRE_V2 CPUs

However, strictly speaking this fixes the SpectreRSB variant of Spectre v2, doesn't it? Shouldn't this be handled as an separately issue? (See #224)

Also, please notice this Red Hat bugzilla comment on bz1616245:

This issue does not affect the versions of the kernel package as shipped with
Red Hat Enterprise Linux 5, 6, 7 and Red Hat Enterprise MRG 2.

However, spectre-meltdown-checker on RHEL 7.5 reports the kernel as vulnerable.

[balonik]

On the same bz new comment explains why RHEL6/7 kernels are not affected by SpectreRSB. Comment #8.

[ChristianGfK]

Seems like Red Hat is claiming their kernels have been doing this for longer than the upstream patch. They don't seem to be using X86_FEATURE_RSB_CTXSW, or priting any relevant output during boot, though?

[ChristianGfK]

Is this even fixable? It would probably mean white-listing RHEL's default combination of mitigiations, somehow?