Mitigating Spectre attacks

[jfoote]

Hello,

You are probably well aware, but some mainstream compilers are emitting retpolines to help mitigate Spectre variant 2 attacks. Do you have any plans to add a similar capability to the cretonne code generator (and/or do you think it makes sense for cretonne to do this sort of thing)?

Thanks, Jon

/cc @tyler @pchickey

[stoklund]

Yes, I expect we will need mitigations for all variants of Spectre. All of these attacks depend on a high-resolution timer being available to malicious code, so a first-level mitigation is to make sure that the available timer resolution is low enough to make the possible exfiltration bandwidth impractical.

Beyond that, I expect the following areas to be affected in Cretonne:

Heap bounds checking

Heaps that have 4 GB of guard pages should not be affected, but heap styles that use less virtual address space will generate the kind of bounds checking code that can be exploited to read the full 4 GB of address space following the heap base. We can implement different styles of mitigation:

• If the heap maximum size is small, we can use a constant index mask to limit the range of memory that can be read by speculative code. See the WebKit blog post about index masking.
• A dynamic index mask may also be appropriate in some cases.
• We can construct a mask using setCC instructions and use that to mask loaded values when code is executed speculatively.
• For ARM ISAs, we can use their recommended conditional move patterns along with the new barrier instruction.

Jump table bounds checking

Cretonne doesn't generate jump tables yet, but it is expected to do so in the future to get good performance for br_table instructions.

The bounds check on a jump table load can be exploited in much the same way as a heap bounds check. The speculatively executed indirect branch would go to an address that was loaded from outside the jump table. This could potentially be a "speculative gadget" that could be used to leak data.

To mitigate this, we can make sure that jump table sizes are a power of two and use index masking to prevent speculative loads from accessing memory outside the jump table.

Independent from that, we may also have to use a "retpoline" instead of an indirect branch to avoid indirect branch predictor aliasing. The retpoline alone does not prevent the bounds check exploit.

Table call bounds checking

WebAssembly indirect calls use a table of function pointers and trap if you attempt to call a function index outside the table. This bounds check can be exploited the same way as a jump table bounds check.

And we need to use retpolines for the indirect call too.

[jfoote]

Thanks for the speedy response Jakob!

Yes, I expect we will need mitigations for all variants of Spectre. All of these attacks depend on a high-resolution timer being available to malicious code, so a first-level mitigation is to make sure that the available timer resolution is low enough to make the possible exfiltration bandwidth impractical.

Agreed -- FWIW, in our discussions we have defined these requirements for the disclosed flush+reload / evict+time attacks on speculative execution (much detail omitted for brevity):

1. Priming the system
2. Problematic speculative execution
3. Measuring side effects
4. Orchestrating the attack

The idea is that if you can holistically prevent any one of these steps in your design, you break the attacks. In terms of forward-looking architectures we might build cretonne into it is difficult to provide assurance that any one of these will not be possible as systems evolve, so in addition to addressing immediate vulnerabilities we are looking at fixes/mitigations in each step.

Heap bounds checking

Great -- this matches our analysis as well. For programs / linear memories without guard pages, I assume going the lfence route would be far too expensive, but we haven't performed tests along these lines.

Jump table bounds checking

Using a mask here to mitigate Spectre variant bytecodealliance/cranelift#1 attacks in cases where makes sense. As I understand cretonne, I think retpoline will be necessary to mitigate Spectre variant bytecodealliance/cranelift#2 attacks in all cases (as you mentioned in a later comment), including abuse of jump tables.

Table call bounds checking

A mitigation (i.e. index masking) or a fix for Spectre variant bytecodealliance/cranelift#1 should be applied here as well then, correct? Maybe this was implied -- just want to make sure I understand correctly.

Jon

[stoklund]

Re: Heap bounds checking

I haven't measured the performance of using lfence to guard bounds checks myself, but from what I hear from people who did, results are very sad.

If an application embedding Cretonne can ensure that a power-of-two sized area of memory following the heap base does not contain sensitive information, I think masking the index is a better solution.

re: Table call bounds checking

Yes, table calls would need mitigation for both Spectre variants 1 and 2 — bounds checking and indirect branch predictor aliasing.

Since retpolines specifically disable the indirect branch predictor, I expect that the scales are tipped in favor of using branch trees instead of jump tables for more instances of the WebAssembly br_table instruction.

[jfoote]

That makes sense to me, thanks Jakob.

This GH issue was meant to be an inquiry -- I'll close it for now. If you'd prefer to use it to track this as a work item feel free to re-open.

[stoklund]

Thanks, Jonathan. I'll leave it open to track remaining work.

[tedwin1113]

Suggested alternate usage scenario: detect only.

No idea how to build this (last time I used an assembler was in 1984 !;), but given these vulnerabilities may never be fixed, seems to me some detection-only tools able to identify live on-line sources of infection would be very helpful. Even more important, some means of detecting tainted package/module updates. (And to be clear, I'm talking about tools to be used on completely unprotected systems.)

As one who has extra non-critical-use machines running 24x7, I'd gladly invest time fitting some of them out with detection tools, even with at a performance penalty. Not sure what a 'gadget' is, but seems like it might be possible to plant some 'special-gadgets' capable of leaving breadcrumbs.(?) Also wonder if a 'whistleblower' exploit could be concocted, run-solo on a clean system while storing profile data, then, following each upgrade, detect interference by a potential unsub, by comparing profile changes.

Thanks to all working this seriously fubar situation.

[cfallin]

I'm doing some issue gardening here and I believe we have done most of the above by now: we have conditional move-based mitigations on heap accesses, table accesses, and branch table loads. We don't have any indirect predictor-specific mitigations yet, but we're working on incorporating hardware CFI techniques (see bytecodealliance/rfcs#17).

I'll leave it up to our processor-vendor folks whether they think we have sufficient mitigations now, or with the CFI features, to mark this issue "resolved" -- @akirilov-arm, @abrown, @uweigand, others, what do you think?

[akirilov-arm]

Concerning the Arm architecture, Arm has published documentation on the various speculative execution vulnerabilities. Unfortunately the hardware CFI techniques that are discussed by the RFC proposal mentioned above do not really help with speculative execution attacks that target indirect branch predictors - they rely on instructions that raise processor exceptions, which in most implementations does not happen while executing speculatively; that is somewhat similar to the straight-line speculation vulnerability covered by the documentation.

As for the attacks that apply to conditional direct branches, e.g. bounds checks, according to Arm's guidance the conditional moves should be followed by the CSDB instruction in order to future-proof the mitigations (that barrier instruction executes as a no-op on processors that predate it).

[akirilov-arm]

Recently I have done a little bit of research on Spectre-V2, and my conclusion is that on AArch64 the only efficient mitigation (so I exclude techniques such as retpolines) is provided by the FEAT_CSV2_2 extension to the Arm architecture. It specifies a system register, SCXTNUM_EL0, that can be used to associate a software-defined context number with branch prediction, and as a result branch predictions for one context are unable to influence speculative execution in another one. So, one implementation approach would be to reserve a context number, e.g. 0, for the WebAssembly runtime, and then come up with a number for each Wasm context that the runtime manages. On a transition from the runtime implementation to a Wasm module, for example, it would be necessary to execute:

msr scxtnum_el0, xn
isb

where Xn holds the context number associated with the Wasm module. Note that no changes to the code generated from WebAssembly are necessary. Depending on the threat model, it might also make sense for each Wasm module to get its own runtime context number (that is, the runtime is not always going to use 0), or even a (pseudo)random value (the PACGA instruction provided by the pointer authentication extension is an efficient way to obtain one). The hardware itself does not assign a special meaning to any context number value.

The FEAT_CSV2_2 extension is supported by recent processors such as Arm Cortex-A510, A710, A715, X2, and X3, and also Arm Neoverse N2. Unfortunately AFAIK the Linux kernel (as of version 6.0-rc5) does not expose an interface such as getauxval(AT_HWCAP2) to query about the availability of that extension.

We also have a similar attack to contend with, Spectre-BHB. The mitigation against it is to execute a small loop that would be located right next to the operation setting SCXTNUM_EL0 - the details are in the white paper that Arm has published about the attack. Future processors may have even more efficient mitigations - the white paper covers those too.