openhwgroup
Add hardware debug support
This PR adds support for RISC-V Debug Specification Version 1.0.0. There are some quirks with privilege unit interactions which I intend to cleanup soon. Otherwise, the RTL is tested and functional.
This PR also integrates some basic JTAG tests into the regression testing system. I'm open to any feedback that can make it better.
Looks very interesting to me. I really enjoyed working on JTAG support on the Tensilica ML605 fpga dev board.
Piet, I'm delighted you've made all this progress with the FPGA board. If you have bandwidth for working on debug mode, starting with developing a test suite, and then fixing any bugs you unearth, we'd be happy to support and we'd love you to be a Wally contributor. Debug mode is an important feature for us, but the last student working on it left and we don't have anybody ramped up on it at the moment.
I pulled the debug mode pull request and looked at it. It's rather large, my roommate works on Risc-v at TensTorent and is familiar with the Risc-v debug extension. I'll email the authors of the debug pull request.
I wasn't able to understand the changes even well enough to know if it will work on my Arty A7-100.
I'm having trouble getting my Arty A7 to boot Linux. It's getting stuck in the zsbl/sd.c code on the 2nd command. The code on the 2nd cmd isn't well documented, nothing on the 4 bits of error. ChatGpt was a bit of help. I'll try emailing the authors.
I wonder if my not hacking the fpga/generator Make file could be wrong. Your forthcoming book on this wasn't clear. Looking at the Makefile I didn't see the need for modifying it.
Perhaps you have updated your draft of your forthcoming book and could send Yuri an update. Maybe it will help me get the Arty FPGA booting.
I supported the Tensilicas Xtensa architecture for a few years. I documented the step by step procedure for using FTAG debugging on the Xilinx ML605. This included full Buildroot support. I was surprised that the Wally build instructions seem to be building the classic Buildroot tools but not installing the Buildroot output to the SD card. The root filesystem is empty and the Makefile didn't appear to be copying the Buildroot output to the SD card filesystem. At tensilica we used cpio to make the device nodes and copy the Linux user space objects.
Sorry it is taking so long to get up to speed with your Wally framework.
Maybe an update on your forthcoming book would help.
-piet
On Mon, Dec 30, 2024, 8:24 PM David Harris @.***> wrote:
Piet, I'm delighted you've made all this progress with the FPGA board. If you have bandwidth for working on debug mode, starting with developing a test suite, and then fixing any bugs you unearth, we'd be happy to support and we'd love you to be a Wally contributor. Debug mode is an important feature for us, but the last student working on it left and we don't have anybody ramped up on it at the moment.
— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/pull/883#issuecomment-2566121562, or unsubscribe https://github.com/notifications/unsubscribe-auth/BKFFREUVRC7OFNWHF5JYLO32IIMBXAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKNRWGEZDCNJWGI . You are receiving this because you commented.Message ID: @.***>
Example ML605 FPGA Support at Tensilica:
https://wiki.linux-xtensa.org/index.php?title=SMP_HiFi_2_Development_Board
I tried merging Matthew-Otto's pull request and got a pre-hook failure. I'm not familiar with this. About to go for a swim, any tips? I'll take a look at this later this evening. Looks like argument mismatches.
I heard the JTAG Debug stuff was going to be continued this summer. Any progress on that front?
===================================================== pre-commit hook failure
[INFO] Checking merge-conflict files only. check for added large files..............................................Passed check for case conflicts.................................................Passed check docstring is first.................................................Passed check for broken symlinks............................(no files to check)Skipped fix requirements.txt.................................(no files to check)Skipped check for merge conflicts................................................Passed check json...........................................(no files to check)Skipped check toml...........................................(no files to check)Skipped check yaml...........................................(no files to check)Skipped ruff.....................................................................Failed
- hook id: ruff
- exit code: 1
bin/wsim:38:1: SyntaxError: Unexpected indentation | 36 | parser.add_argument("--lockstepverbose", "-lv", help="Run ImperasDV lock, step, and compare with tracing enabled", action="store_t⦠37 | parser.add_argument("--jtag", "-j", help="JTAG SVF file to load", default=0) 38 | parser.add_argument("--rvvi", "-r", help="Simulate rvvi hardware interface and ethernet.", action="store_true") | ^ 39 | return parser.parse_args() |
bin/wsim:41:1: SyntaxError: Expected a statement | 39 | return parser.parse_args() 40 | 41 | def validateArgs(args): | ^ 42 | if not args.testsuite and not args.elf: 43 | print("Error: Missing test suite or ELF file") |
bin/wsim:166:1: SyntaxError: Unexpected indentation | 164 | cmd = cd + "; " + prefix + " vsim -do "" + cmd + " +acc -GDEBUG=1"" 165 | cmd = f"do wally.do {args.config} {args.testsuite} {args.tb} {args.args} {args.params} {args.define} {flags}" 166 | cmd = cd + "; " + prefix + " vsim -do "" + cmd + " +acc"" | ^ 167 | print(f"Running Questa with command: {cmd}") 168 | cmd = cd + "; " + prefix + " vsim -c -do "" + cmd + """ |
bin/wsim:168:1: SyntaxError: Unexpected indentation | 166 | cmd = cd + "; " + prefix + " vsim -do "" + cmd + " +acc"" 167 | print(f"Running Questa with command: {cmd}") 168 | cmd = cd + "; " + prefix + " vsim -c -do "" + cmd + """ | ^ 169 | if (args.jtag): 170 | cmd += " +JTAGTESTFILE=" + args.jtag |
bin/wsim:169:5: SyntaxError: Expected a statement | 167 | print(f"Running Questa with command: {cmd}") 168 | cmd = cd + "; " + prefix + " vsim -c -do "" + cmd + """ 169 | if (args.jtag): | ^ 170 | cmd += " +JTAGTESTFILE=" + args.jtag 171 | cmd += """ |
bin/wsim:175:1: SyntaxError: Expected a statement | 173 | os.system(cmd) 174 | 175 | def runVerilator(args): | ^ 176 | print(f"Running Verilator on {args.config} {args.testsuite}") 177 | os.system(f'make -C {WALLY}/sim/verilator WALLYCONF={args.config} TEST={args.testsuite} TESTBENCH={args.tb} PLUS_ARGS="{args.args⦠|
Found 6 errors.
We use a pre-commit check python linter (ruff) to ensure our python scripts are compatible with the wide variety of different python versions used by the different distros we support. That was added after this PR was initially drafted, so it’s likely causing issues. This PR has enough conflicts that it’s probably going to be hard to pull without significant refactoring.
Consider this PR highly experimental. It hasn’t been meaningfully tested, and there’s no reason to believe it is correct yet. If you want to implement your own debug, you could use it for ideas, and we would be happy to collaborate on a test plan.
On Jun 24, 2025, at 10:35 PM, Jordan Carlin @.***> wrote:
jordancarlin left a comment (openhwgroup/cvw#883) https://github.com/openhwgroup/cvw/pull/883#issuecomment-3003391401 We use a pre-commit check python linter (ruff) to ensure our python scripts are compatible with the wide variety of different python versions used by the different distros we support. That was added after this PR was initially drafted, so it’s likely causing issues. This PR has enough conflicts that it’s probably going to be hard to pull without significant refactoring.
— Reply to this email directly, view it on GitHub https://github.com/openhwgroup/cvw/pull/883#issuecomment-3003391401, or unsubscribe https://github.com/notifications/unsubscribe-auth/AR4AA33554ASLWHZBFZXPOD3FIYKHAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBTGM4TCNBQGE. You are receiving this because you commented.
Pete,
Thanks for the Email - sorry for the delay; was traveling. That repo is older and is definitely not correct - I have a newer one but its definitely not correct yet. I am working on a test plan but I would welcome any involvement in this. Let me know if interested. Take care.
All my best,
James
On Jun 24, 2025, at 7:30 PM, Pete Delaney @.***> wrote:
[https://avatars.githubusercontent.com/u/176838802?s=20&v=4]PietDelaney left a comment (openhwgroup/cvw#883)https://github.com/openhwgroup/cvw/pull/883#issuecomment-3002238274
I tried merging Matthew-Otto's pull request and got a pre-hook failure. I'm not familiar with this. About to go for a swim, any tips? I'll take a look at this later this evening. Looks like argument mismatches.
I heard the JTAG Debug stuff was going to be continued this summer. Any progress on that front?
===================================================== pre-commit hook failure
[INFO] Checking merge-conflict files only. check for added large files..............................................Passed check for case conflicts.................................................Passed check docstring is first.................................................Passed check for broken symlinks............................(no files to check)Skipped fix requirements.txt.................................(no files to check)Skipped check for merge conflicts................................................Passed check json...........................................(no files to check)Skipped check toml...........................................(no files to check)Skipped check yaml...........................................(no files to check)Skipped ruff.....................................................................Failed
- hook id: ruff
- exit code: 1
bin/wsim:38:1: SyntaxError: Unexpected indentation | 36 | parser.add_argument("--lockstepverbose", "-lv", help="Run ImperasDV lock, step, and compare with tracing enabled", action="store_t� 37 | parser.add_argument("--jtag", "-j", help="JTAG SVF file to load", default=0) 38 | parser.add_argument("--rvvi", "-r", help="Simulate rvvi hardware interface and ethernet.", action="store_true") | ^ 39 | return parser.parse_args() |
bin/wsim:41:1: SyntaxError: Expected a statement | 39 | return parser.parse_args() 40 | 41 | def validateArgs(args): | ^ 42 | if not args.testsuite and not args.elf: 43 | print("Error: Missing test suite or ELF file") |
bin/wsim:166:1: SyntaxError: Unexpected indentation | 164 | cmd = cd + "; " + prefix + " vsim -do "" + cmd + " +acc -GDEBUG=1"" 165 | cmd = f"do wally.do {args.config} {args.testsuite} {args.tb} {args.args} {args.params} {args.define} {flags}" 166 | cmd = cd + "; " + prefix + " vsim -do "" + cmd + " +acc"" | ^ 167 | print(f"Running Questa with command: {cmd}") 168 | cmd = cd + "; " + prefix + " vsim -c -do "" + cmd + """ |
bin/wsim:168:1: SyntaxError: Unexpected indentation | 166 | cmd = cd + "; " + prefix + " vsim -do "" + cmd + " +acc"" 167 | print(f"Running Questa with command: {cmd}") 168 | cmd = cd + "; " + prefix + " vsim -c -do "" + cmd + """ | ^ 169 | if (args.jtag): 170 | cmd += " +JTAGTESTFILE=" + args.jtag |
bin/wsim:169:5: SyntaxError: Expected a statement | 167 | print(f"Running Questa with command: {cmd}") 168 | cmd = cd + "; " + prefix + " vsim -c -do "" + cmd + """ 169 | if (args.jtag): | ^ 170 | cmd += " +JTAGTESTFILE=" + args.jtag 171 | cmd += """ |
bin/wsim:175:1: SyntaxError: Expected a statement | 173 | os.system(cmd) 174 | 175 | def runVerilator(args): | ^ 176 | print(f"Running Verilator on {args.config} {args.testsuite}") 177 | os.system(f'make -C {WALLY}/sim/verilator WALLYCONF={args.config} TEST={args.testsuite} TESTBENCH={args.tb} PLUS_ARGS="{args.args� |
Found 6 errors.
— Reply to this email directly, view it on GitHubhttps://github.com/openhwgroup/cvw/pull/883#issuecomment-3002238274, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFSIHPBWQKDNG6RFLEJHPQ33FHURPAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBSGIZTQMRXGQ. You are receiving this because you are subscribed to this thread.Message ID: @.***>
Dr. Stine:
I'm very interested in working with you. At tensilica developed a SMP linux and I increased the JTAG debugging speed by 100x by making it like OpenOCD and doing ASYNC communication over the FT2232 jtag to usb adapter. I loved it and very interested in getting Wally able to do the same with RISC-V as we did on Tensilica's Xtensa architecture.
Jordan any suggestions on dealing with this issue?
I was thinking of seeing if ChatGPT had some suggestions and if tying to build would point out the things that needs to be fixed. I don't know how Dr. Stine's group is going to be working on this in the future. Maybe a side branch would be one approach. I do like the kernel's paradigm of debasing commits instead of lots of merging like I did on Tensila's Xtensa git repositories. As I recall AOSP is the same, re-basing instead of a sequence of git merges.
Pete,
Please call me James. Sounds great. I will be traveling soon for a week but free all of the time after 7/11 - interested in a meeting to get together? If you are too busy, I can at least share what I have, if interested. Take care.
All my best,
James
On Jun 26, 2025, at 12:09 AM, Pete Delaney @.***> wrote:
[https://avatars.githubusercontent.com/u/176838802?s=20&v=4]PietDelaney left a comment (openhwgroup/cvw#883)https://github.com/openhwgroup/cvw/pull/883#issuecomment-3007072165
Dr. Stine:
I'm very interested in working with you. At tensilica developed a SMP linux and I increased the JTAG debugging speed by 100x by making it like OpenOCD and doing ASYNC communication over the FT2232 jtag to usb adapter. I loved it and very interested in getting Wally able to do the same with RISC-V as we did on Tensilica's Xtensa architecture.
— Reply to this email directly, view it on GitHubhttps://github.com/openhwgroup/cvw/pull/883#issuecomment-3007072165, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFSIHPHEWCGTJCRZSMDNJG33FN6BXAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBXGA3TEMJWGU. You are receiving this because you commented.Message ID: @.***>
Pete,
Just FYI - we don’t have any working code and anything we have is in an infant state. I just wanted to make sure that things are very early in the stage of development. We would love to have you help and take a leadership role, if possible. I am happy to meet with you when I get back around 7/14. Take care.
All my best,
James
On Jun 26, 2025, at 7:58 AM, Stine, James @.***> wrote:
Pete,
Please call me James. Sounds great. I will be traveling soon for a week but free all of the time after 7/11 - interested in a meeting to get together? If you are too busy, I can at least share what I have, if interested. Take care.
All my best,
James
On Jun 26, 2025, at 12:09 AM, Pete Delaney @.***> wrote:
[https://avatars.githubusercontent.com/u/176838802?s=20&v=4]PietDelaney left a comment (openhwgroup/cvw#883)https://github.com/openhwgroup/cvw/pull/883#issuecomment-3007072165
Dr. Stine:
I'm very interested in working with you. At tensilica developed a SMP linux and I increased the JTAG debugging speed by 100x by making it like OpenOCD and doing ASYNC communication over the FT2232 jtag to usb adapter. I loved it and very interested in getting Wally able to do the same with RISC-V as we did on Tensilica's Xtensa architecture.
— Reply to this email directly, view it on GitHubhttps://github.com/openhwgroup/cvw/pull/883#issuecomment-3007072165, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFSIHPHEWCGTJCRZSMDNJG33FN6BXAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMBXGA3TEMJWGU. You are receiving this because you commented.Message ID: @.***>
James: I live in Silicon Valley so I'm not sure if we will have an chance to meet in the near future. If you do visit Silicon Valley I'm rather sure Yuri would be delighted to talk to you and perhaps you would be interested in his Verilog Meetups what we have on Sundays around noon.
I'm getting more familiar with Matthew-Otto's pull request commit and trying to see if I can at least get it update with the current top of trunk bits.
I'd love to see your latest bits and work together with you when you return from your trip.
=piet
I fixed the lint errors and commited my 1st draft. I'm trying top and fpga builds.
Pushed it to my github cvw repo:
https://github.com/PietDelaney/cvw/tree/piet-30-June-jtag
The cvw.lattest.jtag/src/debug/notes.txt has a good starting point for messing with it on my Arty-A7. perhaps I can use the second ft2232 uart to try it. At tensilica I added the ft2232 chip for our development boards to make it unnecessary to use a dongle. I'm not famiar with the SiPEED dongle. TBD...
Pete,
Thanks for the Email and sorry for the delay. I will try taking a look soon, but going on a vacation starting tomorrow for a week (1st time in 5 years). I appreciate the comments but promise to touch base when I get back. I definitely will involve you with this as we go forward. I want to get this at least defined in the test plan by end of July and hopefully starting to work on the updated implementation. Take care.
All my best,
James
On Jul 1, 2025, at 1:34 AM, Pete Delaney @.***> wrote:
[https://avatars.githubusercontent.com/u/176838802?s=20&v=4]PietDelaney left a comment (openhwgroup/cvw#883)https://github.com/openhwgroup/cvw/pull/883#issuecomment-3022057769
I fixed the lint errors and commited my 1st draft. I'm trying top and fpga builds.
Pushed it to my github cvw repo:
https://github.com/PietDelaney/cvw/tree/piet-30-June-jtag
The cvw.lattest.jtag/src/debug/notes.txt has a good starting point for messing with it on my Arty-A7. perhaps I can use the second ft2232 uart to try it. At tensilica I added the ft2232 chip for our development boards to make it unnecessary to use a dongle. I'm not famiar with the SiPEED dongle. TBD...
— Reply to this email directly, view it on GitHubhttps://github.com/openhwgroup/cvw/pull/883#issuecomment-3022057769, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AFSIHPAECNC4CJ5UCNOAFJ33GITXVAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTAMRSGA2TONZWHE. You are receiving this because you commented.Message ID: @.***>
This is a year old. I am closing it, but we can still refer to it when somebody implements debug.
James: How's your work on the RISC-V DEBUG/JTAG work going?
I've been busy preparing for HIP Surgery. Sigh!
@PietDelaney Oh No on the hip surgery. Hope things go smoothly for a quick and healthy return. Thanks for the Email. It's going great. DMI and DTMCS seem to be working along with a ID code for Wally. DM is a little harder with all its registers and interaction. We have a test plan in place and trying to finish up implementation by tomorrow. We also have several test benches. I am hoping to add a branch to the repo very soon so you can check out.
We also have a simple JTAG version. This version including Boundary Scan so it will be different than the debug module. You are welcome to play with that. Includes a FPGA implementation on Arty A7 and also a Tcl file to import that I created to help creating a project easily in Vivado: https://github.com/stineje/Drop-In-JTAG/tree/main
Hope to see your new debug/jtag branch. I was looking at https://github.com/stineje/Drop-In-JTAG/tree/main and cloned it. Looked interesting.
How's your work on JTAG Debug support going? I've been busy preparing for a hip replacement in November. Do you have a branch in development I could check out and see how you're doing?
Pete,
Thanks for the email and your interest. We currently have a subset of the debug module working with abstract commands for read/write. We have also tested it and it works well. We hope to make a PR and hopefully someone will review soon before it gets in. We also hope to add stuff as we go (e.g., trigger modules).
Hope your surgery goes well as well as a speedy recovery.
Take care.
All my best,
James
On Sep 20, 2025, at 1:55 AM, Pete Delaney @.***> wrote:
PietDelaney left a comment (openhwgroup/cvw#883) https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenhwgroup%2Fcvw%2Fpull%2F883%23issuecomment-3314657858&data=05%7C02%7Cjames.stine%40okstate.edu%7Cabfed78fedb14e64b4c508ddf812aaae%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C638939481252735211%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=nNB7FHh%2FL%2FJiX7j%2BWZGhGQohzRcuXLYSzWsm0J3Xhzo%3D&reserved=0 How's your work on JTAG Debug support going? I've been busy preparing for a hip replacement in November. Do you have a branch in development I could check out and see how you're doing?
— Reply to this email directly, view it on GitHub https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fopenhwgroup%2Fcvw%2Fpull%2F883%23issuecomment-3314657858&data=05%7C02%7Cjames.stine%40okstate.edu%7Cabfed78fedb14e64b4c508ddf812aaae%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C638939481252756344%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=LHkaJ7wz7uclshmo4A7pudhNaUAAJV2nXImHEVPJLdE%3D&reserved=0, or unsubscribe https://nam04.safelinks.protection.outlook.com/?url=https%3A%2F%2Fgithub.com%2Fnotifications%2Funsubscribe-auth%2FAFSIHPCBIO2OXOOOV5ZZAJD3TT25RAVCNFSM6AAAAABK5FCPISVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTGMJUGY2TOOBVHA&data=05%7C02%7Cjames.stine%40okstate.edu%7Cabfed78fedb14e64b4c508ddf812aaae%7C2a69c91de8494e34a230cdf8b27e1964%7C0%7C0%7C638939481252772853%7CUnknown%7CTWFpbGZsb3d8eyJFbXB0eU1hcGkiOnRydWUsIlYiOiIwLjAuMDAwMCIsIlAiOiJXaW4zMiIsIkFOIjoiTWFpbCIsIldUIjoyfQ%3D%3D%7C0%7C%7C%7C&sdata=OoI18odIoT8wFbpjaIqWtpKk5GB2SimfVg8KhvrNhNM%3D&reserved=0. You are receiving this because you commented.