Running tests, spike or qemu ?

[eroom1966]

I apologise in advance if this sounds a dumb question.

Having successfully built the tools, and the simulator. How can I run the ISA tests on a reference simulator ?

For example if I want to run the executable build/isa/rv32ui-p-add how should I do this

I tried spike -d build/isa/rv32ui-p-add but saw an exception error for illegal instruction

core   0: 0xffffffff80000074 (0x3b029073) csrw    pmpaddr0, t0
core   0: exception trap_illegal_instruction, epc 0xffffffff80000074
core   0:           badaddr 0x0000000000000000

in essence I want to run the ISA tests to get disassembly and register dumping from the golden reference

Thx E

[aswaterman]

this is expected behavior - the illegal-instruction trap handler will skip over this instruction

[eroom1966]

Hmm, I will double check but I think it gets stuck in a loop soon afterwards E.

[eroom1966]

@aswaterman You were correct, but a later exception gets stuck into a loop core 0: 0x00000000800000bc (0x00000073) ecall core 0: exception trap_machine_ecall, epc 0x00000000800000bc

This goes back to the trap_vector, and ends up looping forever in

80000040 <write_tohost>:
80000040:	00001f17          	auipc	t5,0x1
80000044:	fc3f2023          	sw	gp,-64(t5) # 80001000 <tohost>
80000048:	ff9ff06f          	j	80000040 <write_tohost>

Rethinking this I think the invocation was incorrect and should have been spike -isa=RV32IMAC -d build/isa/rv32ui-p-add seems the default is 64IMAFDC

if I run in non-debug mode, eg spike -isa=RV32IMAC build/isa/rv32ui-p-add then spike performs a silent exit, is there a way to know the test has passed

In fact what I would like to do is get a step-by-step execution, logging the register changes and instruction disassembly - is there a method to do this ?

Thanks E

[aswaterman]

--enable-commit-log when building spike, I think? I've never used it.

On Tue, Oct 10, 2017 at 5:17 AM eroom [email protected] wrote:

@aswaterman https://github.com/aswaterman You were correct, but a later exception gets stuck into a loop core 0: 0x00000000800000bc (0x00000073) ecall core 0: exception trap_machine_ecall, epc 0x00000000800000bc

This goes back to the trap_vector, and ends up looping forever in

80000040 <write_tohost>: 80000040: 00001f17 auipc t5,0x1 80000044: fc3f2023 sw gp,-64(t5) # 80001000 80000048: ff9ff06f j 80000040 <write_tohost>

Rethinking this I think the invocation was incorrect and should have been spike -isa=RV32IMAC -d build/isa/rv32ui-p-add seems the default is 64IMAFDC

if I run in non-debug mode, eg spike -isa=RV32IMAC build/isa/rv32ui-p-add then spike performs a silent exit, is there a way to know the test has passed

In fact what I would like to do is get a step-by-step execution, logging the register changes and instruction disassembly - is there a method to do this ?

Thanks E

— You are receiving this because you were mentioned.

Reply to this email directly, view it on GitHub https://github.com/riscv/riscv-tests/issues/82#issuecomment-335453624, or mute the thread https://github.com/notifications/unsubscribe-auth/AA-7wphtKXDeTmviTtcaCr65kt-SdrLbks5sq2BOgaJpZM4PyeBB .

[eroom1966]

@aswaterman

Hi Andrew, maybe I am asking the wrong question :-/

I am trying to run the suite of verification tests from https://github.com/riscv/riscv-tests I think I should use spike, as this is the golden reference, would you agree ?

If I run one of the produced tests, eg rv64uc-p-rvc, I run under spike in the following manner spike -l --isa=RV64IMAC --pc=0x80000000 rv64uc-p-rvc

is this the correct method to run under spike ? I cannot find anything that shows how these validation tests should be correctly executed.

I make 2 observations from running the above simulation.

1. implicit prolog It seems that when running the above test, there is an implicit prolog before running the test itself

core   0: 0x0000000000001000 (0x00000297) auipc   t0, 0x0
core   0: 0x0000000000001004 (0x02028593) addi    a1, t0, 32
core   0: 0x0000000000001008 (0xf1402573) csrr    a0, mhartid
core   0: 0x000000000000100c (0x0182b283) ld      t0, 24(t0)
core   0: 0x0000000000001010 (0x00028067) jr      t0
core   0: 0x0000000080000000 (0x04c0006f) j       pc + 0x4c // Now we are in the test itself

I can see what is happening here, it is simply constructing a jump to the entry point, from the command line option --pc=xxxx (in this case 0x80000000), what is the definition of the layout in memory prior to the jump - it would appear that the entry point is placed in memory at location 0x1000 + 24 => 0x1018 is this something hardcoded in spike, where do these prolog instructions get defined ?

2. loop at end It appears at the end of the run, the ecall causes an exception, the trap_vector branches to <write_tohost>, which has an infinite loop

core   0: 0x0000000080000040 (0x00003f17) auipc   t5, 0x3
core   0: 0x0000000080000044 (0xfdcf2023) sw      t3, -64(t5)
core   0: 0x0000000080000048 (0xff9ff06f) j       pc - 0x8 // back to 0x80000040

given the behavior in 1 & 2 above, how would I determine from running the simulation, whether the test had passed or failed ?

Apologies in advance for what may be stupid questions, I am still trying to get to grips with the infrastructure

Thx

[aswaterman]

The prologue is generated here: https://github.com/riscv/riscv-isa-sim/blob/master/riscv/sim.cc#L233

The loop at the end is waiting for an external agent to poll that memory location (the one being stored-to inside that loop). Through a complicated chain of events, the polling happens from here: https://github.com/riscv/riscv-fesvr/blob/master/fesvr/htif.cc#L175

[LeeMooreImperas]

@aswaterman Hi Andrew, can you please explain the correct behavior for the following scenarios

1. Access to a CSR which is defined in the spec but does not exist in a given implementation, e,g, the optional PMPADDR registers

2. Access to a CSR which is undefined in the spec, ie the index is not currently defined.

Does the instruction get treated as a NOP, Cause an exception - and subsequent handler to be called, or something different

for the above, can spike model the necessary behaviors

Thx Lee

[aswaterman]

In both cases, the implementation should raise an illegal-instruction exception. This is what the spec means by "Attempts to access a non-existent CSR raise an illegal instruction exception."

[LeeMooreImperas]

@aswaterman Hi Andrew,

This is what we had understood, but when running under spike we see a write of a pmpaddr register, an exception is raised, but we do not see a handler called - this issue was described earlier in this thread on the 9/Oct. your response was

this is expected behavior - the illegal-instruction trap handler will skip over this instruction

This is surprising, I would expect the handler to be called when the illegal instruction occurs, rather than immediately skipping over the illegal instruction, without calling a handler maybe I am misunderstanding ? Here is a snippet from running spike

core   0: 0xffffffff80002a84 (0x305292f3) csrrw   t0, mtvec, t0
3 0xffffffff80002a84 (0x305292f3) x 5 0xffffffff80000008
core   0: 0xffffffff80002a88 (0x3b071073) csrw    pmpaddr0, a4
core   0: exception trap_illegal_instruction, epc 0xffffffff80002a88
core   0:           badaddr 0x0000000000000000
core   0: 0x0000000080002a90 (0x7fbfd797) auipc   a5, 0x7fbfd
3 0x0000000080002a90 (0x7fbfd797) x15 0xffffffffffbffa90

I am expecting that after the exception (at 80002a88), the trap handler would be called which would then perform some action, but I see the program step onto the next instruction at 0x80002a90, without having called the trap handler - I must be misunderstanding some behavior :-(

Thx Lee

[aswaterman]

I understand the confusion now. It is invoking the handler: at this point in the program, 80002a90 is the handler address. (Note the instruction at 80002a84, which sets the handler address to 80002a90.)

This is the code pattern I use for "execute this instruction if it's legal, but if it's not legal, don't worry about it." I do this by setting mtvec to the address after the potentially illegal instruction, so that the pc advances past the instruction whether or not the trap occurs.

[ForrestBlue]

@eroom1966 Do you solve you issue? I met the same issue as you did

[eroom1966]

As I recall, the handler WAS called, it so happens the handler address is set to the next instruction after the exception address, so it appears to continue running

[ForrestBlue]

@eroom1966 did you see the "PASS" string display ? I can see spike quietly run and nothing display, on tracing, the "add" test eventually successfully executed I find this page but not sure it is helpful or not https://wiki.itap.purdue.edu/display/RISC/Testing+Methodology

0000000080000604 80000604: 0ff0000f fence 80000608: 00100193 li gp,1 8000060c: 00000073 ecall 80000610: c0001073 unimp

[edcote]

@ForrestBlue @eroom1966 @aswaterman ... just hit this as well, do either have updates? I am going to dig a bit now

[edcote]

didn't spike used to issue a test pass indication? my test case passes, same conditions @ForrestBlue

// this is RVTEST_PASS -->
core   0: 0x00000000800000fc (0x0ff0000f) fence
core   0: 0x0000000080000100 (0x00100193) li      gp, 1
core   0: 0x0000000080000104 (0x00000073) ecall
// OK ^^^ -->
core   0: exception trap_user_ecall, epc 0x0000000080000104
[..]

[aswaterman]

Spike returns the error code to the OS when it exits. It's possible some Makefile or script was looking for the exit code and printing a message.

[edcote]

I'm trying to confirm that now ...

1. hack or.S and introduce a failure

+++ b/isa/rv64ui/or.S
@@ -26,7 +26,7 @@ RVTEST_CODE_BEGIN
   # Source/Destination tests
   #-------------------------------------------------------------
 
-  TEST_RR_SRC1_EQ_DEST( 6, or, 0xff0fff0f, 0xff00ff00, 0x0f0f0f0f );
+  TEST_RR_SRC1_EQ_DEST( 6, or, 0xff0fff0f, 0xff00ff00, 0x0f0f0f0e ); // <--- insert error
   TEST_RR_SRC2_EQ_DEST( 7, or, 0xff0fff0f, 0xff00ff00, 0x0f0f0f0f );
   TEST_RR_SRC12_EQ_DEST( 8, or, 0xff00ff00, 0xff00ff00 );

2. rerun tests

$ cd $RISCV/../riscv-tests/isa
$ make clean
$ make run
$ echo $?
0

... without luck (edit: actually, it does work, make clean simply acting weird)

[edcote]

nm, I'll go back and look at why make run returns 0 (I tried twice).

$ spike  rv64ui-v-or
*** FAILED *** (tohost = 6)

edit: works fine now.... (moar coffee please, its going to be a long night)

spike -l --isa=rv64gc rv64ui-p-or 2> rv64ui-p-or.out
Makefile:49: recipe for target 'rv64ui-p-or.out' failed
make: *** [rv64ui-p-or.out] Error 6

[pfaelectrique]

hi , can someone help me to force the --enable-threads i have the following config : when i do /opt/riscv-musl/bin/riscv64-unknown-elf-gcc -v

Using built-in specs. COLLECT_GCC=/opt/riscv-musl/bin/riscv64-unknown-elf-gcc COLLECT_LTO_WRAPPER=/opt/riscv-musl/libexec/gcc/riscv64-unknown-elf/11.1.0/lto-wrapper Target: riscv64-unknown-elf Configured with: /home/local/source/riscv-gnu-toolchain/riscv-gcc/configure --target=riscv64-unknown-elf --prefix=/opt/riscv-musl --disable-shared --disable-threads --enable-languages=c,c++ --with-system-zlib --enable-tls --with-newlib --with-sysroot=/opt/riscv-musl/riscv64-unknown-elf --with-native-system-header-dir=/include --disable-libmudflap --disable-libssp --disable-libquadmath --disable-libgomp --disable-nls --disable-tm-clone-registry --src=.././riscv-gcc --enable-multilib --with-abi=lp64d --with-arch=rv64imafdc --with-tune=rocket 'CFLAGS_FOR_TARGET=-Os -mcmodel=medlow' 'CXXFLAGS_FOR_TARGET=-Os -mcmodel=medlow' Thread model: single Supported LTO compression algorithms: zlib gcc version 11.1.0 (GCC)

i don't know where to change to enable threads thank you

[indumathiraju]

Good afternoon all, i have few doubts regarding this benchmark, I need know 1.why we need riscv-tests what is really purpose of this benchmark.

2.Can riscv-tests ported & executed to FU740 unmatched board riscv64

3. i did execute the debug directory on Hifive1 board using ./gdbserver.py ,why am not able execute on u74 board.

4.what is really use of isa and benchmark directory ,what is correct way execute tests, how to get log of test suite.

please kindly answer these question

Thank you

[indumathiraju]

i got log from Hifive1 board when i execute ./gdbserver.py targets/sifive/Hifive1.py Test: CheckMisa Target: spike32_2 ---------------------------------[ Compile ]----------------------------------

• riscv64-unknown-elf-gcc -g programs/checksum.c programs/tiny-malloc.c programs/infinite_loop.S -DDEFINE_MALLOC -DDEFINE_FREE programs/entry.S programs/init.c -DNHARTS=2 -I ../env -T targets/RISC-V/spike32.lds -nostartfiles -mcmodel=medany -DXLEN=32 -o spike32_2_checksum-40341101 -march=rv32imav -mabi=ilp32 -------------------------[ /tmp/spike-7zcq_kbh.log ]--------------------------
• spike -p2 --isa RV32IMAV --dm-auth --dm-no-halt-groups --varch=vlen:128,elen:64 -m0x10000000:0x10000000 --rbb-port 0 spike32_2_checksum-40341101 Listening for remote bitbang connection on port 35219. warning: tohost and fromhost symbols not in ELF; can't communicate with target

--------------------------------[ Traceback ]--------------------------------- Traceback (most recent call last): File "/home/indumathi/riscv-tests/debug/testlib.py", line 1114, in run self.classSetup() File "/home/indumathi/riscv-tests/debug/testlib.py", line 1166, in classSetup BaseTest.classSetup(self) File "/home/indumathi/riscv-tests/debug/testlib.py", line 1081, in classSetup self.server = self.target.server(self) File "/home/indumathi/riscv-tests/debug/targets.py", line 166, in server return testlib.Openocd(server_cmd=self.server_cmd, File "/home/indumathi/riscv-tests/debug/testlib.py", line 349, in init self.process = self.start(cmd, logfile, extra_env) File "/home/indumathi/riscv-tests/debug/testlib.py", line 353, in start process = subprocess.Popen(cmd, stdin=subprocess.PIPE, File "/usr/lib/python3.8/subprocess.py", line 858, in init self._execute_child(args, executable, preexec_fn, close_fds, File "/usr/lib/python3.8/subprocess.py", line 1704, in _execute_child raise child_exception_type(errno_num, err_msg, err_filename) FileNotFoundError: [Errno 2] No such file or directory: 'openocd' -------------------------[ /tmp/spike-7zcq_kbh.log ]--------------------------

• spike -p2 --isa RV32IMAV --dm-auth --dm-no-halt-groups --varch=vlen:128,elen:64 -m0x10000000:0x10000000 --rbb-port 0 spike32_2_checksum-40341101 Listening for remote bitbang connection on port 35219. warning: tohost and fromhost symbols not in ELF; can't communicate with target

-------------------------------[ End of logs ]-------------------------------- Result: exception Logfile: logs/20220331-125130-spike32_2-CheckMisa.log Reproduce: ./gdbserver.py targets/RISC-V/spike32-2-hwthread.py CheckMisa Time elapsed: 0.39s

what is the use of this test ?