CVA6 (Ariane) not working in Firesim and slow Dhrystone on FPGA

[manox]

I have two problems with CVA6.

The first is that I can't get CVA6 to work in Firesim. Started workloads give no output and exit immediately. Rocket works without any problems.

I have also tested systems generated with Chipyard on an FPGA (VCU118). With Rocket and Boom I also get plausible results here with Dhrystone and Coremark. However, with CVA6, the results for Dhrystone are relatively poor (~ 0.7 DMIPS/Mhz) and depend on the l2 cache (which is not the case with rocket/boom). Coremark is okay (> 2 Coremark/Mhz).

I'm happy for suggestions, thanks.

[alonamid]

What do you mean by "exits immediately" in FireSim? what is the last output you see? Could you also clarify your comment about CVA6 Dhrystone depending on the L2 cache?

[manox]

What do you mean by "exits immediately" in FireSim? what is the last output you see?

Here is the workloads UART log:

Script started on Wed 13 Jan 2021 01:50:09 PM UTC
AFI PCI  Vendor ID: 0x1d0f, Device ID 0xf000
Using xdma write queue: /dev/xdma0_h2c_0
Using xdma read queue: /dev/xdma0_c2h_0
UART0 is here (stdin/stdout).
command line for program 0. argc=52:
+permissive +mm_relaxFunctionalModel_0=0 +mm_openPagePolicy_0=1 +mm_backendLatency_0=2 +mm_schedulerWindowSize_0=8 +mm_transactionQueueDepth_0=8 +mm_dramTimings_tAL_0=0 +mm_dramTimings_tCAS_0=14 +mm_dramTimings_tCMD_0=1 +mm_dramTimings_tCWD_0=10 +mm_dramTimings_tCCD_0=4 +mm_dramTimings_tFAW_0=25 +mm_dramTimings_tRAS_0=33 +mm_dramTimings_tREFI_0=7800 +mm_dramTimings_tRC_0=47 +mm_dramTimings_tRCD_0=14 +mm_dramTimings_tRFC_0=160 +mm_dramTimings_tRRD_0=8 +mm_dramTimings_tRP_0=14 +mm_dramTimings_tRTP_0=8 +mm_dramTimings_tRTRS_0=2 +mm_dramTimings_tWR_0=15 +mm_dramTimings_tWTR_0=8 +mm_rowAddr_offset_0=18 +mm_rowAddr_mask_0=65535 +mm_rankAddr_offset_0=16 +mm_rankAddr_mask_0=3 +mm_bankAddr_offset_0=13 +mm_bankAddr_mask_0=7 +slotid=0 +profile-interval=-1 +macaddr0=00:12:6D:00:00:02 +blkdev0=coremark-bare0-dummy.rootfs +niclog0=niclog0 +blkdev-log0=blkdev-log0 +trace-select=1 +trace-start=0 +trace-end=-1 +trace-output-format=0 +dwarf-file-name=coremark-bare0-coremark.bare.riscv-dwarf +autocounter-readrate=0 +autocounter-filename=AUTOCOUNTERFILE +drj_dtb=coremark-bare0-coremark.bare.riscv.dtb +drj_bin=coremark-bare0-coremark.bare.riscv +drj_rom=coremark-bare0-coremark.bare.riscv.rom +print-start=0 +print-end=-1 +linklatency0=6405 +netbw0=200 +shmemportname0=default +permissive-off coremark-bare0-coremark.bare.riscv 
TraceRV 0: Tracing disabled, since +tracefile was not provided.
random min: 0x0, random max: 0xffffffffffffffff
Commencing simulation.
*** FAILED *** (tohost = 1337)

*** FAILED *** (code = 1337) after 28066711 cycles
time elapsed: 0.3 s, simulation speed = 88.57 MHz
FPGA-Cycles-to-Model-Cycles Ratio (FMR): 1.02
Ran 28066711 cycles (fastest target clock)
[FAIL] FireSim Test
SEED: 1610545809
 at cycle 4294967295
Script done on Wed 13 Jan 2021 01:50:10 PM UTC

Could you also clarify your comment about CVA6 Dhrystone depending on the L2 cache?

The result with (~ 0.7 DMIPS/Mhz) and without (~ 0.4 DMIPS/Mhz) L2 cache differs strongly. This is not the case with Rocket and Boom.

[alonamid]

I tried reproducing your issue using a fresh clone of Chipyard 1.4. With the exception of a build error which is the result of a bug in Chipyard 1.4 (addressed in #782 ), I was not able to reproduce your issue. The system started booting Linux, and did not exit the simulation at the point described in the issue. That being said, the system did not complete booting Linux, but rather entered into a "livelock" situation while starting syslogd, which is an issue that we will look further into (might be because of the Linux version upgrade in Chipyard 1.4). However, this does not seem related to the issue you described. Is it possible you are using a unique workload?

Regarding your second question, note that CVA6 exposes an AXI memory interface rather than a Tilelink memory interfaces. This has implications on it's connection with the Tilelink buses and the L2 memory syste,.

[manox]

I tried reproducing your issue using a fresh clone of Chipyard 1.4. With the exception of a build error which is the result of a bug in Chipyard 1.4 (addressed in #782 ), I was not able to reproduce your issue.

Thanks, I'll have to test it again with tag 1.4. I used commit f387c4b before.

Is it possible you are using a unique workload?

I tested Coremark and br-base Linux.

Regarding your second question, note that CVA6 exposes an AXI memory interface rather than a Tilelink memory interfaces. This has implications on it's connection with the Tilelink buses and the L2 memory syste,.

Yes, I suspected something like that.

[michael-etzkorn]

That being said, the system did not complete booting Linux, but rather entered into a "livelock" situation while starting syslogd, which is an issue that we will look further into (might be because of the Linux version upgrade in Chipyard 1.4).

This seems to be the issue I experienced. ./init started running, but never executed any instructions.

[alonamid]

@michael-etzkorn , this should have been fixed in #909