[rv_dm] V1 Signoff

[msfschaffner]

trafficstars

Description

Ensure V1 signoff criteria are fulfilled after focus area changes have landed.

[rswarbrick]

I've gone through this reasonably carefully. I'm not completely sure whether we can follow the light-weight signoff here, but hopefully this information will be useful either way.

Commits since Earlgrey-ES tapeout

Checked with version c31cc3247c (the head of master on 23/2/24). There are lots of commits since the Earlgrey ES tapeout, found by git log Earlgrey-M2.5.2-RC0..HEAD --oneline hw/ip/rv_dm.

• d6b5a48011 [RV_DM]rv_dm_abstractcmd_status_vseq
• 39ce8f45d6 [RV_DM]rv_dm_abstractcmd_testentry
• 882c862068 [RV_DM]rv_dm_progbuf_busy_vseq
• b2d81a3b9b [rv_dm,dv] Shuffle tasks in smoke vseq
• 0fa0d6041e [rv_dm,dv] Factor out cases in smoke vseq
• 4fff7d214e [rv_dm,dv] Add a short wait in rv_dm_smoke_vseq
• 71af5e75f0 [rv_dm,dv] fix typo in seq
• 7c3cb2c3a7 [RV_DM]rv_dm_mem_tl_access_resuming_vseq
• df543ebd4d [rv_dm,dv] Remove TODOs when setting reset values in env cfgs
• c690338f41 [RV_DM]rv_dm_mem_tl_access_halted_vseq
• baf655517f [rv_dm,dv] Add checks to randomization in rv_dm_tap_fsm_vseq
• 253a68801c [rv_dm,dv] Add checks to randomization in ..._debug_disabled_vseq
• 091f292034 [rv_dm,dv] Add a trivial clocking block to rv_dm_if
• c88d92587d [RV_DM]rv_dm_scoreboard
• 85a73504ed [RV_DM]rv_dm_stress_all_vseq
• 74c1b11679 [rv_dm,dv] Remove a TODO in rv_dm_env.core
• ab4b36f6b9 [ipgen,rstmgr] Fix paths to rstmgr in md files
• fc8484601e [reggen,hw] Create index parameter for registers windows
• 31433a46e2 [RV_DM]rv_dm_stress_all_vseq
• 7849658cc9 [RV_DM]rv_dm_mem_tl_access_resuming_vseq
• 18f1692fb6 [rv_dm,dv] Slightly strengthen dmi_debug_disabled test
• 331ca9e1ea [rv_dm,dv] Add comments to rv_dm_jtag_dmi_debug_disabled_vseq
• 48cb2e2a83 [RV_DM]rv_dm_mem_tl_access_halted_vseq
• de31bdf1c2 [reggen] Remove the devmode input
• c535a58f13 [RV_DM] rv_dm_jtag_dtm_hard_reset_vseq
• 637d1b88e1 [doc,reset] Fix references to reset behavior
• 5a92acbcdf [RV_DM] rv_dm_jtag_dtm_hard_reset_vseq
• 0dc1bca205 [rv_dm,dv] update test description for jtag_dtm_hard_reset
• fde2e0c057 [RV_DM]JTAG_dtm_idle_hint_vseq
• 358d838e3a [rv_dm,dv] fix typo in the testplan
• 51130d721c [RV_DM] rv_dm_jtag_dmi_debug_disabled_vseq
• 17d386c879 [rv_dm,dv] fix regression failure
• ee62ddbf54 [rv_dm/doc] Add some missing CSR documentation
• ca2b62b28a [dv, testplan] Replace descr by desc for consistency
• 6329668bc0 [rv_dm,sival] RV_DM post silicon validation testplan
• f2591927dd [RV_DM]jtag_dmi_dm_inactive
• 4143382e7e [RV_DM]jtag_dtm_idle_hint
• 5d8bab5473 [RV_DM]ndmreset_req sequence
• aa2af96865 [RV_DM]halt_resume_whereto
• efea9f8468 [rv_dm,dv] Add whereto check to halt_resume testplan
• 467dfb2168 [rv_dm,dv] update testplan for halt/going/resume
• 1b16ca2122 [reggen] Add mubi support SWAccess that sets/clears a reg
• bb5f3b61d4 [rv_dm] update register description
• 7aeb4c2676 [RV_DM]Dataaddr_rw_access Test
• 6585d934a0 [RV_DM] Updated HALTED and RESUMING test
• 9cf0ef2b6b [RV_DM] test_rename
• 21b873302c [RV_DM]cmderr_halt_resume_test
• 806815a183 [RV_DM] Hart unavailable test
• 59f8142826 [doc] Moved badges over to using hosted images
• 0773985f85 [RV_DM] Hart resuming test
• a23f5d6d7d [rv_dm] included seq to seq list
• 0eaa70fbb0 [rv_dm] remove extra displays
• 50d9dbb5fb [rv_dm] Halted test
• 7d5f36f4e3 [RV_DM] Command error exception test
• e9ba8b38fa [RV_DM]cmderr_not_supported_test
• ee6f00f5db [doc] rv_dm registers and interfaces now use CMDGEN
• a0f850982d [RV_DM]cmderr_busy_test
• 420c29a1f3 [RV_DM]cmderr_busy test
• 7688e714e8 [reggen] Add initial support for version and cip_id hjson fields
• fbd888eea8 Revert "[reggen] Add CIP_IDs and bump all major versions"
• 0ba10b3cd3 [reggen] Add CIP_IDs and bump all major versions

This is a long list! But most of it is DV implementation work done by Imparé. Looking at the other changes first (and ignoring the reverted CIP_ID change), I find:

• ab4b36f6b9 [ipgen,rstmgr] Fix paths to rstmgr in md files
  • Fixing doc typo
• fc8484601e [reggen,hw] Create index parameter for registers windows
  • Adds (unused) parameter
• de31bdf1c2 [reggen] Remove the devmode input
  • No change in behaviour
• 637d1b88e1 [doc,reset] Fix references to reset behavior
  • Documentation change (not affecting things that are tracked in V1)
• ee62ddbf54 [rv_dm/doc] Add some missing CSR documentation
  • Documentation change, documenting some undocumented CSRs.
• ca2b62b28a [dv, testplan] Replace descr by desc for consistency
  • Minor tooling tidyup
• 6329668bc0 [rv_dm,sival] RV_DM post silicon validation testplan
  • SiVal testplan, which won't affect V1
• 1b16ca2122 [reggen] Add mubi support SWAccess that sets/clears a reg
  • No change in behaviour
• 59f8142826 [doc] Moved badges over to using hosted images
  • Documentation infrastructure change.
• ee6f00f5db [doc] rv_dm registers and interfaces now use CMDGEN
  • Documentation infrastructure change.
• 7688e714e8 [reggen] Add initial support for version and cip_id hjson fields
  • Only changes hjson file. No change to design or DV code.

None of those changes should affect the V1 status.

There are quite a lot of DV changes, driven by the work that Imparé has been doing. I've tried to narrow down a bit to things that might affect V1 status.

I think the V1 checklist items that might be affected are:

• TESTPLAN_COMPLETED

  This is actually the closest to being affected by the changes that have gone in. Looking at the history of hw/ip/rv_dm/data/rv_dm_testplan.hjson find lots of changes. These all add testpoints.

  The background is that RV_DM was signed off as V1 status back in 2022, but we never did much DV work after that. Imparé have started trying to improve coverage numbers (as reported by nightly regressions), which has required adding extra tests/testpoints. This explains the testplan changes that have happened.

• SIM_ALT_TOOL_SETUP

  I've checked, and this seems to work on both Xcelium and VCS.

• TESTPLAN_REVIEWED

  I'm not sure how this should work. An earlier version of the testplan was reviewed, and it's not completely obvious whether changes to the testplan should invalidate this checklist item.

  Given how this has been driven by coverage closure concerns, it might make sense to consider this to still be complete.

Issues closed since the Earlgrey-ES tapeout

Using the filter is:issue is:closed closed:>2023-06-27 rv_dm finds quite a lot of issues. I've dropped the issues that were closed as not-planned (after checking that they aren't really relevant here).

• #21531 Unrelated DV change (I mentioned rv_dm in one of the comments)
• #20700 Documentation bug. Fixed by 637d1b88e1 above.
• #19044 All action items have been spun into separate issues.
• #18635 Unrelated SW bug (which mentions rv_dm in description)
• #18291 Chip-level discussion (mentioning rv_dm in one comment)
• #16689 CDC check tracking issue
• #13648 Chip-level test. Closed as obsolete.
• #10462 lc_ctrl issue which mentions rv_dm as another way of getting some information out.
• Issues tracking implementation of tests:
  • #17036
  • #17035
  • #17033
  • #15670
  • #15667

Currently open issues

Here is a list of tests based on the filter is:issue is:open rv_dm NOT sival NOT chip-test (to exclude sival tracking issues). I've skipped the V1/D2 signoff issues.

• #21517
  • Minor DV testbench problem. Fix pending in a PR.
• #20829
  • This tracks implementation of an RFC (approved).
  • The implementation will be in rv_dm and this is not yet done (23/2/24)
• #19750
  • Potential design change which is not yet implemented (23/2/24)
  • An implementation will only have a small DV impact (tweak scoreboard; plan and implement a vseq to misuse the register).
• #19375
  • SW only
• #18395
  • TODO tracking. This needn't gate V1 signoff (and some items have actually been implemented already).
• #18310
  • A task for V2 tracking. This needn't gate V1 signoff.
• #17034
  • Tracking implementation of a test (which might well turn out to be a situation that can't occur and doesn't need testing).
• #16148
  • An issue in a different block (which mentions rv_dm as an example of how to do it!)
• #15803
  • Chip-level DV problem
• #15668
  • Unimplemented test. This is tracked in the testplan, so shouldn't gate V1.
• #15123
  • Low(ish) priority potential design change.
  • Our (currently incorrect) behaviour isn't actually tracked in models.
  • We'd probably want to improve the model/scoreboard side if we implement the correct behaviour, but it won't need explicit testplanning.
• #14345
  • Tooling issue (mentioning rv_dm as somewhere that worked around the problem).

Throwing away the items that can probably be ignored for V1, I believe these are the only two tasks that might affect V1 signoff:

• #20829
• #19750

Coverage report from 21/02/2024

All V1 tests are implemented. Looking more closely at the report, they also all pass for most seeds: it's not like there's one test which is completely broken.

Summary

• I don't believe that the lightweight sign-off flow really makes sense for rv_dm to get to V1.
• Fortunately, I think we are ready to do a full sign-off meeting.
• I propose we schedule one soon.

[rswarbrick]

We had a V1 review meeting this afternoon. Notes here.

The meeting was happy that we have satisfied the items on the V1 checklist, except for the fact that we hadn't actually addressed V2_CHECKLIST_SCOPED (my fault!). Below are some rough estimates for the work I expect V2 signoff to entail.

@andreaskurth, @hcallahan-lowrisc: Would you mind taking a look at this? If you agree with the rough estimates below, I think we're ready to sign off rv_dm with a V1 status.

---

The V1 checklist contains an item, V2_CHECKLIST_SCOPED. This ensures that we've thought through any pending DV work that we know is going to be needed for V2 signoff.

Some of the V2 checklist items are necessarily dependent on what happens between now and then(!) but here are the items that we currently know will require work, together with rough estimates for how many days' work they might entail.

• DV_DOC_COMPLETED

  The current DV document still has some placeholder items! I think they are probably just examples that can be deleted, but there's some tidying required.

  Estimate: 1 day

• FUNCTIONAL_COVERAGE_IMPLEMENTED

  The existing coverage plan lies at the bottom of rv_dm_testplan.hjson. It's pretty barebones, and there's not currently much implemented (e.g. rv_dm_env_cov.sv is empty)

  Estimate: 4 days

• SIM_ALL_TESTS_PASSING

  Obviously this is pretty hard to estimate! But most existing tests do currently pass, so we might be hopeful this isn't actually too much work.

  Estimate: 5 days

• SIM_FUNCTIONAL_COVERAGE_V2

  This is also pretty hard to estimate. Practically speaking, we're not expecting to have an enormous list of coverpoints and the state space for a debug module isn't enormous.

  Estimate: 5 days.

Summing the numbers above: ~15 days work.

[hcallahan-lowrisc]

I think your estimates seem reasonable to me @rswarbrick. Based on that, I think we should be okay to sign it off. Thanks for keeping it moving!

[andreaskurth]

Thanks @rswarbrick, effort estimates LGTM

[rswarbrick]

Great, thanks guys.