Read Performance Counters: Step by step guide?

[kosterz96]

Hello, I am new into OpenSBI and I am trying to figure out a way so that I can easily access performance counters. I have successfully mapped my rocket-chip design into Alveo U250 using this repository. I tried enabling performance counters by modifying the following files:

• u-boot/arch/riscv/cpu/cpu.c (lines 98-114)

if (CONFIG_IS_ENABLED(RISCV_MMODE)) { /* * Enable perf counters for cycle, time, * and instret counters only */ #ifdef CONFIG_RISCV_PRIV_1_9 csr_write(CSR_MSCOUNTEREN, GENMASK(5, 0)); csr_write(CSR_MUCOUNTEREN, GENMASK(5, 0)); csr_write(CSR_MHPMEVENT3, 0x0802); csr_write(CSR_MHPMEVENT4, 0x1002); #else csr_write(CSR_MCOUNTEREN, GENMASK(5, 0)); csr_write(CSR_SCOUNTEREN, GENMASK(10, 0)); csr_write(CSR_MHPMEVENT3, 0x0802); csr_write(CSR_MHPMEVENT4, 0x1002); #endif

• opensbi/lib/sbi/sbi_hart.c (lines 54-96)

/* Disable user mode usage of all perf counters except default ones (CY, TM, IR) */ if (misa_extension('S') && sbi_hart_has_feature(scratch, SBI_HART_HAS_SCOUNTEREN)){ csr_write(CSR_SCOUNTEREN, -1); }

/**

• OpenSBI doesn't use any PMU counters in M-mode.
• Supervisor mode usage for all counters are enabled by default
• But counters will not run until mcountinhibit is set. */ if (sbi_hart_has_feature(scratch, SBI_HART_HAS_MCOUNTEREN)){ csr_write(CSR_MCOUNTEREN, -1); csr_write(CSR_MHPMEVENT3, 0x0802); csr_write(CSR_MHPMEVENT4, 0x1002); }

Changing those files did not result in enabling performance counters, instead I keep getting an Illegal Instruction error when I try to access hpmcounters using a C program similar to this. Of course reading cycles counter has no problem, as it is enabled by default.

Navigating through the OpenSBI documentation brought me to this, which seems kinda relevant but I can't locate the device tree. Also, I read about event_idx codes and so, in this doc, but I can't exactly figure out what I have to do. I see there is a struct sbi_pmu_hw_event in opensbi/lib/sbi/sbi_pmu.c but I don't know how I can make something out of it.

So my question is: how can I access performance counters using a simple C program? Using perf is not necessary to me.

[avpatel]

@atishp04 can you check this ?

[xydas97]

Any update my friend @atishp04 @avpatel ?

[atishp04]

@xydas97 : Sorry for not responding earlier. It slipped through cracks for some reason. Are you facing the same problem now also ? Are you hacking U-boot or Is you application running in S-mode ? I see the user space program is quite old too.

I guess you don't have mcountinhibit support ? So you just want to read/write hpmcounters ?

[xydas97]

@atishp04 I am actually running a fedora Linux on fpga with Firesim software. The kernel boots with opensbi and I would like to set the performance counters to read ITLB, DTLB and L2 TLB misses. Currently I modified the opensbi/lib/sbi/sbi_init.c file where I set to zero the bits of mcountinhibit that corresponds to the counters I want to use and I added the following code:

csr_write(CSR_MCOUNTEREN, -1); csr_write(CSR_SCOUNTEREN, -1); csr_write(CSR_MHPMEVENT3, 0x0802); csr_write(CSR_MHPMEVENT4, 0x1002);

and then I read the mhomevent to check if the correct value is set but the value is still 0. sbi_printf(CSR_MHPMEVENT3: %lx 'n", csr_read(CSR_MHPMEVENT3);

Any idea on what might cause the problem and if the above lines do not work, do I need to set the performance counters with the sbi_pmu.c functions? And if so what are the correct steps to follow?

Thank you

[atishp04]

Which softcore are you using ? Does it support CSR_MHPMEVENT3 ? You can confirm the counters supported from the boot log in OpenSBI. After the banner you should see something like:

Boot HART MHPM Info : 8 (0x000007f8)

Since you are booting fedora, perf should work as well if you provide the correct device tree node. https://github.com/riscv-software-src/opensbi/blob/master/docs/pmu_support.md

[xydas97]

Softcore Used: alveo_u250_firesim_rocket_singlecore_no_nic
MHPM Info: Boot HART MHPM Count : 0

Indeed there is no MHPM counters enbaled. Any idea how to do it?

[atishp04]

That means it is not enabled in the softcore running in the FPGA. You need to enable hpmcounters there. Once it is done, you need to do following additional things to use it via perf.

1. specify the counter to event mappings in the device tree PMU nde for OpenSBI to understand what are the correct values need to go into hpmevent.

Here are the binding details https://github.com/riscv-software-src/opensbi/blob/master/docs/pmu_support.md

Here is an example of sifive core: https://github.com/torvalds/linux/blob/master/Documentation/devicetree/bindings/perf/riscv%2Cpmu.yaml

dTLB-load-misses and dTLB-store-misses are standard PMU events described in perf. The DT node is required to map them to hpmevent values.

2. In order to decode other perf event name, you need to create the json file for rocket core. Here is an example of DT node and corresponding json file: https://lore.kernel.org/lkml/[email protected]/T/

[atishp04]

Closing this issue as we had some more private email exchange with more clarifications. Please reopen if you still need some help.