Feature request: end simulation when max cycles reached

[SihaoLiu]

I find that current timeout machnism (which is based on number of idle cycles) does make sense. However, I am still wondering if there is a way to set the max cycles which can prevent the simulation running for too long.

Basically, I wonder can we have some ways to:

1. Know the current cycle time during the simulation like:

dut.clock.getCurrentCycle()

2. End the simulation if max cycles reached:

dut.clock.setMaxCycles(<the maximum number of cycles>)

3. Make it different in the output For those simulation end normally, we have something like following in the output:

...
[info] ScalaTest
[info] Run completed in ... seconds.
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] All tests passed.
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: ...

Maybe we can differ those simulations that end because of max cycles reached in the output like following, but still treat the test as succeeded?

[info] ScalaTest
[info] Run completed in ...
[info] Total number of tests run: 1
[info] Suites: completed 1, aborted 0
[info] Tests: succeeded 1, failed 0, canceled 0, ignored 0, pending 0
[info] End the simulation becuase max cycles reached, All tests passed before it ends.  // like here
[info] Passed: Total 1, Failed 0, Errors 0, Passed 1
[success] Total time: ...

[ducky64]

I'm curious about the use case for needing an absolute maximum cycles: are you encountering livelock, or is it some artifact of your test structure?

Getting cycle counts is something that has been asked for. It's likely that if implemented, it will return the number of cycles since some start time (when you asked for the count to start) - the current architecture makes absolute cycle counts on arbitrary clocks difficult.

For reporting: the success / fail / cancel / ignored / pending is handled by ScalaTest. We might be able to change the categorization of timeouts, but I'm not sure it will be easy (or possible?) to add a whole new category.

[SihaoLiu]

I'm curious about the use case for needing an absolute maximum cycles: are you encountering livelock, or is it some artifact of your test structure?

Getting cycle counts is something that has been asked for. It's likely that if implemented, it will return the number of cycles since some start time (when you asked for the count to start) - the current architecture makes absolute cycle counts on arbitrary clocks difficult.

For reporting: the success / fail / cancel / ignored / pending is handled by ScalaTest. We might be able to change the categorization of timeouts, but I'm not sure it will be easy (or possible?) to add a whole new category.

Thank you for your reply @ducky64

For use case, I think it can be benefitial for both cases. For livelock detection it is obvious. However in my own case, it is more like some artifact of my test structure.

So I have an universial test vector for a group of hardware. And for some of them it will end relatively soon but for others, it will take quite a long time to finish (it is just too long, not livelock, which prevent me from starting a new verification work). For sure I can create a unique testbench for each kind of hardware but the problem is that I have too many kinds of hardware and I still want to use the same test vector.

I don't think it is urgent or critical feature. But to me, it sounds good to have it.

[ducky64]

So I would generally treat a test that overran its maximum cycles as a failed test (assume it failed from livelock), that's a reasonable addition (though I don't think it makes sense to set a default there, so it would be purely opt-in). But your explanation suggests that you want some intermediate state between passed and failed as "terminated early", which I can see the motivation for, but I don't think generalizes well.

I think what you're actually trying to do is have some sanity checks that run fast, and also have a set of tests that are much longer and run infrequently. Consider tagging your tests if using ScalaTest to define a set of long and short tests. That being said, I don't have a good way of marking portions of tests as long or short.

[SihaoLiu]

Yeah, I agree with you. So how do you actually terminate the test that overran its macimum cycles. Is there a way to do that in current chisel-testers2? with both treadle and vcs backend support?

[ducky64]

Sorry, there isn't yet. I think this could be a relatively simple change to implement, but there's a train of major refactor PRs undergoing review, so it might be some time before we can start making feature additions.

[SihaoLiu]

I see thank you