verilator icon indicating copy to clipboard operation
verilator copied to clipboard

Vpi dump module test case

Open AndrewNolte opened this issue 1 year ago • 7 comments

I'm hoping to use this to measure progress on vpi_iterate, vpi_scan, and scope support. I copied some of the iteration logic from cocotb

Related cocotb requests:

  • https://github.com/cocotb/cocotb/pull/3664
  • https://github.com/cocotb/cocotb/pull/3624

Depends on https://github.com/verilator/verilator/pull/4896

AndrewNolte avatar Jan 18 '24 23:01 AndrewNolte

I'm not sure if this is "ready to merge" - are you ready, and did it pass with other simulators?

wsnyder avatar Jan 30 '24 03:01 wsnyder

Oh I didn't even realize there was infra to run with other simulators. I got icarus to run, is there a way to do separate golden output files for different sims? It's saying it passed. I'd also like to test xcleium, but it doesn't look like vpi is set up to run with --nc ?

AndrewNolte avatar Jan 30 '24 17:01 AndrewNolte

All except distro (--dist) tests run with Verilator (--vlt or --vltmt).

It depends on the specific test if it also runs against icarus (-iv), --vcs, --nc, --xsim, etc. --vcs and --iv tend to support the most.

For VPI tests we really need it to pass against at least one other simulator (and encourage more than one), as it's too hard to interpret the IEEE spec and know it's correct otherwise.

So I guess the summary is, the test might not run against --nc but can be changed to.

We don't currently have any golden files for other simulators. Usually I try to make sure the results don't vary between simulators by testing the returned results in C code, as opposed to comparing golden output.

wsnyder avatar Jan 30 '24 17:01 wsnyder

I think the vpi outputs vary so much across simulators, I'm not sure it's worth trying to make one "correct" file. We want them all to ideally converge on a similar interpretation of the vpi. I'm trying to use this .out file to measure that for Verilator, so I was curious if we could also measure other simulator's interpretations here. Icarus is very different for example, and it looks like vpiFullName just returns the vpiName

AndrewNolte avatar Jan 31 '24 18:01 AndrewNolte

I think there's actually quite a bit of code that needs to be added for xcelium to work, since they don't follow the vlog_startup_routines pattern, I can try and add that to this test

AndrewNolte avatar Jan 31 '24 18:01 AndrewNolte

I'm fine with just one alternate.

wsnyder avatar Jan 31 '24 19:01 wsnyder

I modified driver.pl to support xrun, changed the test around to not have so many defines. It was sort of trying to be both a main file and a vpi stub, so I split those entirely. I did this by copying over the verilator.cpp from https://github.com/cocotb/cocotb/ as the main, and adding support for that in driver.pl as well. now we can run this with --xrun and --iv to compare the outputs.

AndrewNolte avatar Feb 12 '24 21:02 AndrewNolte

I'll try and fix that, but I'm not on my usual laptop this week. Also it looks like that license thing has issues with the test, should I just change it back? https://github.com/verilator/verilator/actions/runs/8020255683/job/21909951043?pr=4838

AndrewNolte avatar Feb 27 '24 18:02 AndrewNolte

^ I was able to fix the copyright by putting it test_regress/t/ actually

AndrewNolte avatar Feb 27 '24 18:02 AndrewNolte

@wsnyder I think this PR is good to go, as well as https://github.com/verilator/verilator/pull/4913 and https://github.com/verilator/verilator/pull/4918 on the stack. Apologies for the delay last week, I was out of town

AndrewNolte avatar Mar 05 '24 16:03 AndrewNolte

I guess --xrun isn't being run in CI because that would fail, do I need to do something to get --iv to run in CI?

AndrewNolte avatar Mar 05 '24 16:03 AndrewNolte

I don't want to start testing iv in the CI, as I don't want to have to debug + support failures there. Instead we can manually run iv/xrun/etc as we edit tests, depending what licenses the user has, as currently.

wsnyder avatar Mar 05 '24 17:03 wsnyder