[Arcilator] Performance Issue

[owlxiao]

Hi everyone! I’m glad that the circt community has contributed the arcilator. I’ve been trying to integrate arcilator into my project recently but found that it’s not as fast as advertised.

I've created a project called rtl-sim-benchmark, which is designed to test the speed performance of existing RTL simulators across different benchmarks. Currently, I have integrated arcilator, verilator-1 (single thread), and verilator-2 (multi-thread). In all three test sets (nutshell, riscv-mini, rocketchip), arcilator isn't as well as verilator-1. Here is the automatically generated report from rtl-sim-benchmark.

For the tests, I used this simple approach:

auto Start = std::chrono::system_clock::now();

for (int i = 0; i < Cycles; ++i) {
    dut.set_clock(0);
    dut.eval()

    dut.set_clock(1);
    dut.eval();
}

auto End = std::chrono::system_clock::now();

I just count the cycles needed, measure the time from start to end, and collect some performance data. There’s no interaction with any interfaces; it's just a lot of eval() calls.

However, when I ran the arc-tests, the results were unexpectedly different. Arcilator was actually faster than Verilator. What happened here? Could there be something wrong with my tests?

$ make run
build/small-v1.6/rocket-main ../benchmarks/dhrystone.riscv 
loading segment at 60000000 (virtual address 60000000)
loading segment at 80000000 (virtual address 80000000)
entry 80000000
loaded 20888 program bytes
Microseconds for one run through Dhrystone: 799
Dhrystones per Second:                      1250
mcycle = 399986
minstret = 192528
Benchmark run successful!
----------------------------------------
412458 cycles total
vtor: 54371 Hz
arcs: 527592 Hz