Booting linux with two cores

[daksith]

I am trying to get linux booting with two cores. I have changed the NumCores to 2 in the config. Is this not enough or does more have to be done in the bootrom, zsl etc?

[aitesam961]

I am facing the same issue. Some data points are as follows:

• NumCores parameter was changed to 2 in cheshire_cfg_t DefaultCfg https://github.com/pulp-platform/cheshire/blob/9b4c222df72f74f90ab9f36d80ba7b527f92e62b/hw/cheshire_pkg.sv#L571
• Dual Core cheshire does'nt fit on Genesys2 fabric therefore was ported to VCU118.
• Using OpenOCD + GDB, I can run baremetal .elf on each core. I select the desired core by changing coreid in the openocd config script https://github.com/pulp-platform/cheshire/blob/9b4c222df72f74f90ab9f36d80ba7b527f92e62b/util/openocd.common.tcl#L16
• I built the Linux Image using standard CVA6-SDK flow provided by cheshire. The same image works on the single core cheshire though.
• In the UART console, I do not even see ZSBL prints. It is all blank.

I am wondering if I need to change anything in the image, dts, bootrom, zsbl etc in order to make it work on dual core system. Any clues and pointers are highly appreciated

[anythingtemp8]

I have tried dual core and encountered almost the same results. I used OpenOCD + GDB to debug and figured out the bootrom stops at $pc=0x2000126 when boot mode is set to 2 as I am booting from QSPI flash of VCU118 board using provided STARTUPE3 IP. I am not really sure what is the reason for bootrom to stop instead of going forward and loading ZSL. Maybe someone from the developers would step in to guide.

[bantierr]

So, I did a bit of digging, and I got 2 cores to run. I am sorry for the PR hopes @anythingtemp8, my assumption was really wrong, and core 0 was executing, but core 1 just hanged in the bootloader. Here are the changes i used to get dual-core programs to run (with the default configs):

diff --git a/cheshire.mk b/cheshire.mk
index f22f363..7d22525 100644
--- a/cheshire.mk
+++ b/cheshire.mk
@@ -87,7 +87,7 @@ $(CHS_ROOT)/hw/regs/cheshire_reg_pkg.sv $(CHS_ROOT)/hw/regs/cheshire_reg_top.sv:
        $(REGTOOL) -r $< --outdir $(dir $@)

 # CLINT
-CLINTCORES ?= 1
+CLINTCORES ?= 2
 include $(CLINTROOT)/clint.mk
 $(CLINTROOT)/.generated:
        flock -x $@ $(MAKE) clint && touch $@
diff --git a/hw/cheshire_pkg.sv b/hw/cheshire_pkg.sv
index 4e31078..ca678f5 100644
--- a/hw/cheshire_pkg.sv
+++ b/hw/cheshire_pkg.sv
@@ -568,7 +568,7 @@ package cheshire_pkg;
     Cva6ExtCieLength  : 'h2000_0000,  // [0x2.., 0x4..) is CIE, [0x4.., 0x8..) is non-CIE
     Cva6ExtCieOnTop   : 0,
     // Harts
-    NumCores          : 1,
+    NumCores          : 2,
     CoreMaxTxns       : 8,
     CoreMaxTxnsPerId  : 4,
     CoreUserAmoOffs   : 0, // Convention: lower AMO bits for cores, MSB for serial link
diff --git a/hw/rv_plic.cfg.hjson b/hw/rv_plic.cfg.hjson
index 865be3a..b809069 100644
--- a/hw/rv_plic.cfg.hjson
+++ b/hw/rv_plic.cfg.hjson
@@ -8,7 +8,7 @@
     instance_name: "rv_plic",
     param_values: {
         src: 58,
-        target: 2,  // We need *two targets* per hart: M and S modes
+        target: 4,  // We need *two targets* per hart: M and S modes
         prio: 7,
         nonstd_regs: 0  // Do *not* include these: MSIPs are not used and we use a 64 MiB address space
     },
diff --git a/sw/include/regs/clint.h b/sw/include/regs/clint.h
index 86548b1..277aa20 100644
--- a/sw/include/regs/clint.h
+++ b/sw/include/regs/clint.h
@@ -14,19 +14,27 @@
 extern "C" {
 #endif
 // Number of cores
-#define CLINT_PARAM_NUM_CORES 1
+#define CLINT_PARAM_NUM_CORES 2

 // Register width
 #define CLINT_PARAM_REG_WIDTH 32

 // Machine Software Interrupt Pending  (common parameters)
 // Machine Software Interrupt Pending
-#define CLINT_MSIP_REG_OFFSET 0x0
-#define CLINT_MSIP_P_0_BIT 0
-#define CLINT_MSIP_RSVD_0_MASK 0x7fffffff
-#define CLINT_MSIP_RSVD_0_OFFSET 1
-#define CLINT_MSIP_RSVD_0_FIELD \
-  ((bitfield_field32_t) { .mask = CLINT_MSIP_RSVD_0_MASK, .index = CLINT_MSIP_RSVD_0_OFFSET })
+#define CLINT_MSIP_0_REG_OFFSET 0x0
+#define CLINT_MSIP_0_P_0_BIT 0
+#define CLINT_MSIP_0_RSVD_0_MASK 0x7fffffff
+#define CLINT_MSIP_0_RSVD_0_OFFSET 1
+#define CLINT_MSIP_0_RSVD_0_FIELD \
+  ((bitfield_field32_t) { .mask = CLINT_MSIP_0_RSVD_0_MASK, .index = CLINT_MSIP_0_RSVD_0_OFFSET })
+
+// Machine Software Interrupt Pending
+#define CLINT_MSIP_1_REG_OFFSET 0x4
+#define CLINT_MSIP_1_P_1_BIT 0
+#define CLINT_MSIP_1_RSVD_1_MASK 0x7fffffff
+#define CLINT_MSIP_1_RSVD_1_OFFSET 1
+#define CLINT_MSIP_1_RSVD_1_FIELD \
+  ((bitfield_field32_t) { .mask = CLINT_MSIP_1_RSVD_1_MASK, .index = CLINT_MSIP_1_RSVD_1_OFFSET })

 // Machine Timer Compare
 #define CLINT_MTIMECMP_LOW0_REG_OFFSET 0x4000
@@ -34,6 +42,12 @@ extern "C" {
 // Machine Timer Compare
 #define CLINT_MTIMECMP_HIGH0_REG_OFFSET 0x4004

+// Machine Timer Compare
+#define CLINT_MTIMECMP_LOW1_REG_OFFSET 0x4008
+
+// Machine Timer Compare
+#define CLINT_MTIMECMP_HIGH1_REG_OFFSET 0x400c
+
 // Timer Register Low
 #define CLINT_MTIME_LOW_REG_OFFSET 0xbff8

[anythingtemp8]

Hi @bantierr , Thank you very much. I applied the diff you mentioned and re-generated the bitstream. Unfortunately boot failed and nothing shows on the terminal again. I had reverted the changes you mentioned in https://github.com/pulp-platform/cheshire/pull/194 A quick question: Do I still need to keep the changes mentioned in #194 ? Thanks again...

[bantierr]

hey @anythingtemp8, sorry to hear it didn't work. No, you should definitely not keep the changes of #194. I am not sure why it didn't work. Are you sure you recompiled everything properly ? I am running bare-metal programs using QuestaSim. If you have access to it, you should maybe try running a bare-metal program and look at the traces to see if a core still hangs.

I got the changes from a branch of the project, which disabled the BusErr option, maybe you can try disable it.

Good luck

[anythingtemp8]

Thanks @bantierr , Is the branch you are using, available somewhere online for me to try? I can run baremetal programs on Hardware using JTAG but Linux boot fails. EDIT: Yes I recloned the repo and added the diff you provided. Generated .bit and linux image and flashed to the board. Thanks again.....

[bantierr]

Sure, it's on the repo: branch

I didn't try it tho