gemmini custom config build failed

[mbelda]

Hello,

I am trying to generate a custom config for Gemmini.

On gemmini/src/main/scala/gemmini/ConfigsFP.scala I have added the following:

//FP32 Single Precision Configuration
  val FP32Mesh8HighPerfConfig = defaultFPConfig.copy(inputType = Float(8, 24), spatialArrayOutputType = Float(8, 24), accType = Float(8, 24),
                                               tile_latency = 2,
                                               meshRows = 8, meshColumns = 8,
                                               ex_read_from_acc = false, ex_write_to_spad = false,
                                               mvin_scale_args = Some(ScaleArguments((t: Float, u: Float) => t * u, 4, Float(8, 24), -1, identity = "1.0", c_str="((x) * (scale))")),
                                               mvin_scale_acc_args = Some(ScaleArguments((t: Float, u: Float) => t * u, 4, Float(8, 24), -1, identity = "1.0", c_str="((x) * (scale))")),
                                              )

//===========FP32 8x8 HighPerf=========
class GemminiFP32Mesh8HighPerfConfig extends Config((site, here, up) => {
  case BuildRoCC => Seq(
      (p: Parameters) => {
        implicit val q = p
        implicit val v = implicitly[ValName]
        LazyModule(new Gemmini(GemminiFPConfigs.FP32Mesh8HighPerfConfig))
    }
  )
  case SystemBusKey => up(SystemBusKey).copy(beatBytes = 16)
})

I have also added this config to the RocketConfigs.scala of chipyard and an analogous one to BoomConfigs.scala:

class FP32Mesh8HighPerfGemminiRocketConfig extends Config(
  new gemmini.GemminiFP32Mesh8HighPerfConfig ++                         // use FP32Gemmini systolic array GEMM accelerator
  new freechips.rocketchip.subsystem.WithNBigCores(1) ++
  new chipyard.config.AbstractConfig)

Then I modified the gemmini/configs/GemminiCustomConfigs.scala to include this line:

val FP32Mesh8HighPerfConfig = GemminiFPConfigs.FP32Mesh8HighPerfConfig

And then in that file I changed the customConfig value to that one.

Finally, I went to firesim/deploy and generated the recipe for the build:

firesim-rocket-gemminiFP32Mesh8HighPerf-no-nic-l2-llc4mb-ddr3-40MHz:
    DESIGN: FireSim
    TARGET_CONFIG: DDR3FRFCFSLLC4MB_WithDefaultFireSimBridges_WithFireSimHighPerfConfigTweaks_chipyard.FP32Mesh8HighPerfGemminiRocketConfig
    PLATFORM_CONFIG: MCRams_F40MHz_BaseF1Config
    deploy_triplet: null
    post_build_hook: null
    metasim_customruntimeconfig: null
    bit_builder_recipe: bit-builder-recipes/f1.yaml

But when I try to build it it fails with the following error:

[localhost] out: [info] running chipyard.Generator --target-dir /home/centos/firesim/sim/generated-src/f1/FireSim-DDR3FRFCFSLLC4MB_WithDefaultFireSimBridges_WithFireSimHighPerfConfigTweaks_chipyard.FP32Mesh8HighPerfGemminiBoomConfig-MCRams_F40MHz_BaseF1Config --name firesim.firesim.FireSim.DDR3FRFCFSLLC4MB_WithDefaultFireSimBridges_WithFireSimHighPerfConfigTweaks_chipyard.FP32Mesh8HighPerfGemminiBoomConfig --top-module firesim.firesim.FireSim --legacy-configs firesim.firesim:DDR3FRFCFSLLC4MB_WithDefaultFireSimBridges_WithFireSimHighPerfConfigTweaks_chipyard.FP32Mesh8HighPerfGemminiBoomConfig
[localhost] out: (3,List(UInt<7>(123)))
[localhost] out: L2 InclusiveCache Client Map:
[localhost] out: 	0 <= blkdev-tracker0
[localhost] out: 	1 <= serial-tl
[localhost] out: 	2 <= Core 0 DCache
[localhost] out: 	3 <= Core 0 ICache
[localhost] out: 	4 <= stream-reader
[localhost] out: 	5 <= stream-writer
[localhost] out: 
[localhost] out: [error] (run-main-0) java.lang.reflect.InvocationTargetException
[localhost] out: [error] java.lang.reflect.InvocationTargetException
[localhost] out: [error] 	at ... ()
[localhost] out: [error] 	at freechips.rocketchip.stage.phases.PreElaboration.$anonfun$transform$1(PreElaboration.scala:36)
[localhost] out: [error] 	at ... ()
[localhost] out: [error] 	at ... (Stack trace trimmed to user code only. Rerun with --full-stacktrace to see the full stack trace)
[localhost] out: [error] Caused by: java.lang.IllegalArgumentException: requirement failed

I attach the full log. 2022-11-29--22-48-49-buildafi-Y723XAT3ZMPNJPWM.log

For info, I am working on: Linux ip-192-168-5-162.ec2.internal 3.10.0-1160.66.1.el7.x86_64 #1 SMP Wed May 18 16:02:34 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux LSB Version: :core-4.1-amd64:core-4.1-noarch Distributor ID: CentOS Description: CentOS Linux release 7.9.2009 (Core) Release: 7.9.2009 Codename: Core FireSim Version: 1.14.2-0-g8c5416c0 Gemmini version: commit c47cb7f3eb5c18390f176f3a53c43c8546d487d2 (HEAD, tag: v0.6.3)

[hngenc]

Can you copy-paste the exact line in PreElaboration.scala which fails? I tried to track it, but I ended up with this line, which doesn't seem to have a require or assert statement. I'm wondering if your PreElaboration.scala looks different.

[mbelda]

I have the same line. My entire PreElaborations.scala looks like this.

// See LICENSE.SiFive for license details.

package freechips.rocketchip.stage.phases

import chisel3.RawModule
import chisel3.stage.ChiselGeneratorAnnotation
import firrtl.AnnotationSeq
import firrtl.options.Viewer.view
import firrtl.options.{Dependency, Phase, PreservesAll, StageOptions}
import freechips.rocketchip.config.{Field, Parameters}
import freechips.rocketchip.diplomacy._
import freechips.rocketchip.stage.RocketChipOptions
import freechips.rocketchip.util.HasRocketChipStageUtils

case object TargetDirKey extends Field[String](".")

/** Constructs a generator function that returns a top module with given config parameters */
class PreElaboration extends Phase with PreservesAll[Phase] with HasRocketChipStageUtils {

  override val prerequisites = Seq(Dependency[Checks])
  override val dependents = Seq(Dependency[chisel3.stage.phases.Elaborate])

  override def transform(annotations: AnnotationSeq): AnnotationSeq = {

    val stageOpts = view[StageOptions](annotations)
    val rOpts = view[RocketChipOptions](annotations)
    val topMod = rOpts.topModule.get

    val config = getConfig(rOpts.configNames.get).alterPartial {
      case TargetDirKey => stageOpts.targetDir
    }

    val gen = () =>
      topMod
        .getConstructor(classOf[Parameters])
        .newInstance(config) match {
          case a: RawModule => a
          case a: LazyModule => LazyModule(a).module
        }

    ChiselGeneratorAnnotation(gen) +: annotations
  }

}

[hngenc]

Oh, sorry, I missed this in your attached log:

2022-11-29 22:50:01,353 [flush       ] [INFO ]  [localhost] out: [error] Caused by: java.lang.IllegalArgumentException: requirement failed
2022-11-29 22:50:01,353 [flush       ] [INFO ]  [localhost] out: [error] 	at scala.Predef$.require(Predef.scala:268)
2022-11-29 22:50:01,353 [flush       ] [INFO ]  [localhost] out: [error] 	at boom.ifu.ICacheModule.<init>(icache.scala:144)

So this seems to be an odd issue with BOOM.

Looking above, I think you're trying to build a config with Rocket, rather than one with Boom, right? What if you just delete your changes to BoomConfigs.scala for now to see if your new Rocket config builds then?

If that works, we can go back and try to fix the BOOM config as well.

[mbelda]

The Rocket config builds successfully so the problem must be with BOOM. I need the BOOM to work too because I am trying to compare the execution time of different Gemmini configurations with Rocket and BOOM.

[jerryz123]

The problem was a configuration issue, not a RTL issue. A fix is inflight. I'll update this when resolved

[mbelda]

How is the fix going?

[hngenc]

This should be fixed by now. You might want to try the dev branch of Gemmini, which works with Chipyard 1.9.1 (which has lots of other fixes/improvements as well as this). The installation instructions are here.

[elatalhm]

Hi,

Could we please get some details on what the config changes to Boom were? Or what the problem was exactly? I'm working on a fork of an older version of Boom/Chipyard (Chipyard 1.8.1) so I can't pull all the changes to 1.9.1. If someone can point me to the relevant commits that would be great! Thank you.