chipyard icon indicating copy to clipboard operation
chipyard copied to clipboard

JTAG on VCU118

Open ksungkeun84 opened this issue 1 year ago • 11 comments

Background Work

Chipyard Version and Hash

Release: 1.10.0 Hash: b7644b2

OS Setup

Ex: Output of uname -a + lsb_release -a + printenv + conda list

uname -a Linux sk84 6.5.0-15-generic #15~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Jan 12 18:54:30 UTC 2 x86_64 GNU/Linux

lsb_release -a No LSB modules are available. Distributor ID: Ubuntu Description: Ubuntu 22.04.3 LTS Release: 22.04 Codename: jammy

Other Setup

I have commented out WithNoDebug in fpga/src/main/scala/vcu118/Configs.scala. Then, run the command below in fpga directory:

cd fpga
make SUB_PROJECT=vcu118  bitstream

Current Behavior

Exception occurs while running chipyard.fpga.vcu118.VCU118FPGATestHarness.RocketVCU118Config.fir. Please see the full stack trace below.

Expected Behavior

I'm not sure if JTAG is available in VCU118 yet. I think if board's config does not have WithNoDebug, it supports JTAG. However, VCU118 does include WithNoDebug and the exception occurs when I build the bistream after removing that config.

My questions are: Is JTAG not available yet? If it is available, how can I set or modify the code? If it is not available, what's your plan for it?

Other Information

IOCells generated by IOBinders: IOBinder for interface testchipip.CanHavePeripheryCustomBootPin generated: 1 X GenericDigitalInIOCell IOBinder for interface freechips.rocketchip.devices.debug.HasPeripheryDebug generated: 3 X GenericDigitalInIOCell 1 X GenericDigitalOutIOCell

Total generated 5 IOCells: 4 X GenericDigitalInIOCell 1 X GenericDigitalOutIOCell Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: Unused at scala.Predef$.require(Predef.scala:337) at chipyard.fpga.vcu118.VCU118FPGATestHarnessImp.success(TestHarness.scala:122) at chipyard.harness.WithSimDebug$$anonfun$$lessinit$greater$18.$anonfun$apply$38(HarnessBinders.scala:254) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at chisel3.WhenContext.(When.scala:143) at chisel3.when$.apply(When.scala:38) at chipyard.harness.WithSimDebug$$anonfun$$lessinit$greater$18.$anonfun$new$52(HarnessBinders.scala:254) at scala.collection.immutable.List.map(List.scala:246) at scala.collection.immutable.List.map(List.scala:79) at chipyard.harness.WithSimDebug$$anonfun$$lessinit$greater$18.$anonfun$new$51(HarnessBinders.scala:247) at chipyard.harness.WithSimDebug$$anonfun$$lessinit$greater$18.$anonfun$new$51$adapted(HarnessBinders.scala:245) at chipyard.harness.HarnessBinder$$anonfun$$lessinit$greater$2$$anonfun$apply$2.$anonfun$applyOrElse$1(HarnessBinders.scala:57) at chipyard.harness.HarnessBinder$$anonfun$$lessinit$greater$2$$anonfun$apply$2.$anonfun$applyOrElse$1$adapted(HarnessBinders.scala:50) at chipyard.harness.ApplyHarnessBinders$.$anonfun$apply$1(HarnessBinders.scala:41) at chipyard.harness.ApplyHarnessBinders$.$anonfun$apply$1$adapted(HarnessBinders.scala:40) at scala.collection.IterableOnceOps.foreach(IterableOnce.scala:575) at scala.collection.IterableOnceOps.foreach$(IterableOnce.scala:573) at scala.collection.AbstractIterable.foreach(Iterable.scala:933) at chipyard.harness.ApplyHarnessBinders$.apply(HarnessBinders.scala:40) at chipyard.harness.HasHarnessInstantiators.$anonfun$instantiateChipTops$7(HasHarnessInstantiators.scala:86) at chipyard.harness.HasHarnessInstantiators.$anonfun$instantiateChipTops$7$adapted(HasHarnessInstantiators.scala:85) at scala.collection.immutable.List.foreach(List.scala:333) at chipyard.harness.HasHarnessInstantiators.$anonfun$instantiateChipTops$6(HasHarnessInstantiators.scala:85) at scala.runtime.java8.JFunction0$mcV$sp.apply(JFunction0$mcV$sp.scala:18) at chisel3.withClockAndReset$.apply(MultiClock.scala:26) at chipyard.harness.HasHarnessInstantiators.instantiateChipTops(HasHarnessInstantiators.scala:85) at chipyard.harness.HasHarnessInstantiators.instantiateChipTops$(HasHarnessInstantiators.scala:75) at chipyard.fpga.vcu118.VCU118FPGATestHarnessImp.instantiateChipTops(TestHarness.scala:93) at chipyard.fpga.vcu118.VCU118FPGATestHarnessImp.(TestHarness.scala:127) at chipyard.fpga.vcu118.VCU118FPGATestHarness.module$lzycompute(TestHarness.scala:90) at chipyard.fpga.vcu118.VCU118FPGATestHarness.module(TestHarness.scala:90) at chipyard.fpga.vcu118.VCU118FPGATestHarness.module(TestHarness.scala:24) at freechips.rocketchip.stage.phases.PreElaboration.$anonfun$transform$1(PreElaboration.scala:38) at chisel3.Module$.do_apply(Module.scala:53) at chisel3.stage.phases.Elaborate.$anonfun$transform$2(Elaborate.scala:40) at chisel3.internal.Builder$.$anonfun$build$1(Builder.scala:884) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59) at chisel3.internal.Builder$.build(Builder.scala:879) at chisel3.stage.phases.Elaborate.$anonfun$transform$1(Elaborate.scala:40) at scala.collection.immutable.List.flatMap(List.scala:293) at scala.collection.immutable.List.flatMap(List.scala:79) at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:28) at chisel3.stage.phases.Elaborate.transform(Elaborate.scala:21) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15) at firrtl.options.Translator.transform(Phase.scala:248) at firrtl.options.Translator.transform$(Phase.scala:248) at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15) at firrtl.options.DependencyManager.$anonfun$transform$5(DependencyManager.scala:280) at firrtl.Utils$.time(Utils.scala:181) at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:280) at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183) at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179) at scala.collection.immutable.List.foldLeft(List.scala:79) at firrtl.options.DependencyManager.transform(DependencyManager.scala:269) at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255) at firrtl.options.PhaseManager.transform(DependencyManager.scala:443) at chisel3.stage.ChiselStage.run(ChiselStage.scala:45) at firrtl.options.Stage$$anon$1.transform(Stage.scala:43) at firrtl.options.Stage$$anon$1.transform(Stage.scala:43) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15) at firrtl.options.Translator.transform(Phase.scala:248) at firrtl.options.Translator.transform$(Phase.scala:248) at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15) at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47) at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183) at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179) at scala.collection.immutable.List.foldLeft(List.scala:79) at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47) at logger.Logger$.$anonfun$makeScope$2(Logger.scala:137) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59) at logger.Logger$.makeScope(Logger.scala:135) at firrtl.options.Stage.transform(Stage.scala:47) at firrtl.options.Stage.transform(Stage.scala:17) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15) at firrtl.options.Translator.transform(Phase.scala:248) at firrtl.options.Translator.transform$(Phase.scala:248) at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15) at firrtl.options.DependencyManager.$anonfun$transform$5(DependencyManager.scala:280) at firrtl.Utils$.time(Utils.scala:181) at firrtl.options.DependencyManager.$anonfun$transform$3(DependencyManager.scala:280) at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183) at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179) at scala.collection.immutable.List.foldLeft(List.scala:79) at firrtl.options.DependencyManager.transform(DependencyManager.scala:269) at firrtl.options.DependencyManager.transform$(DependencyManager.scala:255) at firrtl.options.PhaseManager.transform(DependencyManager.scala:443) at chisel3.stage.ChiselStage.run(ChiselStage.scala:45) at firrtl.options.Stage$$anon$1.transform(Stage.scala:43) at firrtl.options.Stage$$anon$1.transform(Stage.scala:43) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:38) at firrtl.options.phases.DeletedWrapper.internalTransform(DeletedWrapper.scala:15) at firrtl.options.Translator.transform(Phase.scala:248) at firrtl.options.Translator.transform$(Phase.scala:248) at firrtl.options.phases.DeletedWrapper.transform(DeletedWrapper.scala:15) at firrtl.options.Stage.$anonfun$transform$5(Stage.scala:47) at scala.collection.LinearSeqOps.foldLeft(LinearSeq.scala:183) at scala.collection.LinearSeqOps.foldLeft$(LinearSeq.scala:179) at scala.collection.immutable.List.foldLeft(List.scala:79) at firrtl.options.Stage.$anonfun$transform$3(Stage.scala:47) at logger.Logger$.$anonfun$makeScope$2(Logger.scala:137) at scala.util.DynamicVariable.withValue(DynamicVariable.scala:59) at logger.Logger$.makeScope(Logger.scala:135) at firrtl.options.Stage.transform(Stage.scala:47) at firrtl.options.Stage.execute(Stage.scala:58) at firrtl.options.StageMain.main(Stage.scala:71) at chipyard.Generator.main(Generator.scala)

ksungkeun84 avatar Feb 06 '24 06:02 ksungkeun84

@jerryz123 Hi Jerry, I've seen your PR #1712 that add JTAG to Arty100T. Do you have any plan for ading JTAG to VCU118? Or could you guide me to do it myself if possible?

ksungkeun84 avatar Feb 06 '24 07:02 ksungkeun84

I don't have immediate plans for this, but I would be happy to review PRs on this.

Does the VCU118 expose an additional JTAG interface (not the one for programming the FPGA)?

jerryz123 avatar Feb 12 '24 19:02 jerryz123

@jerryz123 Thanks for your response. Yes it exposes an extra jag interface for debugging and there is a code for jtag harness binder in FPGA-shell. However, that didn’t become a part of rtl code for some reason. I managed to make jtag debugging possible, do you want me to PR for it?

ksungkeun84 avatar Feb 14 '24 00:02 ksungkeun84

Yes, a PR would be greatly appreciated.

jerryz123 avatar Feb 14 '24 00:02 jerryz123

Keep in mind, that default JTAG interface is used for bitstream download and communicating with Xilinx debug cores. The current offered overlays in fpga-shells are done over the male PMODs which is less invasive and it'd be better to support an additional debug JTAG in a manner which doesn't break a very common workflow of downloading bitstreams and debugging with the ILA.

https://github.com/chipsalliance/rocket-chip-fpga-shells/blob/main/src%2Fmain%2Fscala%2Fshell%2Fxilinx%2FPeripheralsVCU118Shell.scala#L156-L160

Sungkeun Kim @.***> 于 2024年2月14日周三 上午8:46写道:

@jerryz123 https://github.com/jerryz123 Thanks for your response. Yes it exposes an extra jag interface for debugging and there is a code for jtag harness binder in FPGA-shell. However, that didn’t become a part of rtl code for some reason. I managed to make jtag debugging possible, do you want me to PR for it?

— Reply to this email directly, view it on GitHub https://github.com/ucb-bar/chipyard/issues/1785#issuecomment-1942915829, or unsubscribe https://github.com/notifications/unsubscribe-auth/AVF2DIZKG7UYPTU7QQ77OZTYTQCPJAVCNFSM6AAAAABC3MNGRSVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMYTSNBSHEYTKOBSHE . You are receiving this because you are subscribed to this thread.Message ID: @.***>

michael-etzkorn avatar Feb 14 '24 03:02 michael-etzkorn

@michael-etzkorn Thanks for your comment. I'll keep that in mind while I make PR.

ksungkeun84 avatar Feb 16 '24 00:02 ksungkeun84

I have a question for @jerryz123 or @michael-etzkorn. I'm consued with HarnessBinder in chipyard and Overlay in fpga-shells. What this feature (pin mapping for JTAG DEBUG) should be in between them?

If it should be an overlay, where should I put the source between fpga-shell or chipyard? This is because I've seen both HarnessBinder and some custom overlays in chipyard repo.

ksungkeun84 avatar Feb 16 '24 03:02 ksungkeun84

It looks like JTAGDebugVCU118PlacedOverlay is already declared within the TestHarness. This overlay does not take any design input, so you would just call place to hook up to the pins on the board. Shell Input is currently mapping this to FMC, but we should consider PMOD male since I imagine that's easier to set-up and doesn't necessarily rely on a specific mezzanine card's pin mapping.

This handles FPGA pin mapping:

val jtagPlacedOverlay = dp(JTAGDebugOverlayKey).head.place(JTAGDesignInput())

Now, the non-intuitive part of this overlay is that unlike UART and SDIO, it doesn't take any bundle bridge as input and instead gives an IO output through overlayOutput. It also returns ModuleValue[FlippedJTAGIO] which makes me think the intent was to hook up within the implementation of the module itself, though I'm only guessing what the intended API is. At any rate, we should be able to grab jtagPlacedOverlay's overlayOutput and get the IO back.

In VCU118's HarnessBinders, you could do:

class hbWithJtagDebug extends HarnessBinder({
  case (vcu118th: VCU118FPGATestHarnessImp, port:  JTAGPort , chipId: Int)
=> {
               val jtag_io =
vcu118th.outer_.jtagPlacedOverlay.overlayOutput.jtag.getWrappedValue
               // hook up jtag_io to port here
               // treat this like connecting io directly within the
VCU118TestHarness Implementation.
        }
    } }
  }
})

I'll emphasize that this is just a general idea of how the HarnessBinder, ShellPlacer, and Diplomacy APIs.

A JTAG port on the VCU118 will a need for an electrical solution since SDIO occupies female PMOD. You could replace that, but that'd involve a whole new boot process, so finding a way to communicate over FMC or the male PMOD would be more harmonious.

Here's some of the code I looked at for reference.

VCU118TestHarness JTAG overlay declaration: https://github.com/ucb-bar/chipyard/blob/main/fpga/src/main/scala/vcu118/TestHarness.scala#L33

JTAGDebugVCU118PlacedOverlay: https://github.com/chipsalliance/rocket-chip-fpga-shells/blob/main/src/main/scala/shell/xilinx/VCU118NewShell.scala#L244-L268

JtagOverlay https://github.com/chipsalliance/rocket-chip-fpga-shells/blob/main/src/main/scala/shell/JTAGDebugOverlay.scala#L43-L44

VCU118 pins for reference. [image: image.png] [image: image.png]

michael-etzkorn avatar Feb 16 '24 10:02 michael-etzkorn

I don't suspect images from Email will show up on Github.

Pages 90 and 91 for PMOD pins. https://docs.xilinx.com/v/u/en-US/ug1224-vcu118-eval-bd

michael-etzkorn avatar Feb 16 '24 10:02 michael-etzkorn

Hi @michael-etzkorn. Thanks for your response. I'm getting clear about fpga-shells structure. I've updated my code based on your suggestion and I can connect GDB through jtag and openocd. I used FMC module and USB adapter to connect the FPGA bord to the host PC.

I'd like to start PR process but I'd like to know if there is a process to test such as regression test before PR. I've checked CONTRIBUTING.md but it does not have anything about testing. Would you provide any information about it?

ksungkeun84 avatar Feb 19 '24 05:02 ksungkeun84

@jerryz123 and @michael-etzkorn, I've made PR #1796. Would you please review it?

ksungkeun84 avatar Feb 23 '24 07:02 ksungkeun84