Support for simulating precompiled artifacts

[solsjo]

VUnit does a great job of deriving the dependency tree for the compile phase. This relieves the developer of the burden of maintaining a buildsystem.

But there are benefits with allowing a buildsystem to relieve VUnit of that work as well. For instance, the build system bazel + surrounding tools, enables artifacts to be built, cached and tested remotely. The benefit being that if an artifact already exists remotely it won't have to be rebuilt locally. If rebuilt the (ghdl linking) or update of the lib / library.cfg file is only a small increment and a fast operation. If a bazel test-rule depends on an artifact that hasn't changed then the test won't need to run, conversely only a tests with a dependency change needs to be tested. This reduces the amount of tests needed to run during e.g. a commit hook or CI pipeline for a pull request.

For this to work VUnit cannot be allowed to know the whole dependency tree. Rather it only needs the

• test-bench file to parse
• the python test configuration file
• the test artifact

In the extension I could write bazel rules for ghdl compilation and vunit tests.

I have prepared a commit and a draft pull-request which describes the intension, but I'm sure there are more correct ways of handling it.

I'd be happy to contribute and can make the pull request myself if I get feedback on how it should be added. I'm also perfectly fine with someone else doing it, the code change in my commit is pretty small. (test are missing though, as this is still just at the idea stage)

[umarcor]

Hi @solsjo! That sounds interesting! Are you aware of hdl/bazel_rules_hdl? The purpose of that repository is gathering Bazel recipes for HDL|EDA related tools. It would be interesting to gather rules for GHDL compilation and for running VUnit in there.

Other than that, I really like the idea, specially for sharing test results of large open source projects. However, I'm not sure about VUnit providing enough info for being able to reuse test artifacts. In order to be sure about pre-compiled artifacts being reusable, two conditions need to be met:

• The artifacts themselves are readable on different machines. With GHDL, the .cf might be reusable, but .o objects (generated by LLVM and GCC backends) are not portable. Therefore, artifacts need to be identified by the compiler/platform, apart from VUnit's own parameters.
• VUnit provides enough info about the project, the parameters and the tests to uniquely identify a "case". I think this is possible, just maybe challenging to implement.

[solsjo]

Hi! Yes, I’ve browsed hdl/bazel_rules_hdl previously. At that point I didn’t understand or find that many rules. But I’m guessing that they gather them as external repositories via the WORKSPACE file?

Regarding the test re-usability you are right. The .o files won’t be portable. And currently, even if they where it would not help (to my best knowledge). Bazel figures out which host platform it executes on and uses that information to select the correct toolchain (for instance a .exe file in the windows case and makes assumptions about the path format). The host platform won’t change during invocation meaning that you can’t start an invocation on a Mac/windows machine and continue the parallel remote execution on a remote node that is a Linux node. (https://github.com/bazelbuild/bazel/issues/5309). This might be resolved, or there might be ways around it (I have not investigated it much further). But as long as the host platform is the same, re-use I possible. So an oversimplification on my side. But at least Bazel is aware of the artifacts dependency. Which reduces the risk of inflicting self injury by re-using an artifact that might be incompatible.

Other than that there is a bit of work related to figuring out how ghdl handles input in different invocations. It would have been nice if .o files could be turned into .a files and statically link larger libs along the dependency graph similar to cc_library. And take advantage of that ghdl allows supplying compile and link options. But maybe that isn’t possible if ghdl want to reanalyze vhd source files when ghd -e is called. A later problem I guess =) And I guess that each simulators compilation output has its own quirks. Riviera PRO,for instance, updates the library output file for each compiled vhd file part of the lib. Which Bazel wouldn't allow if you don't compile the whole library in a single rule.

The vunit part is simpler. The tests will only need to be re-run if the vunit py test config, the tb or a dependency of those has changed. But the granularity will sort of have to be one py config per tb.*vhd at least if you want to avoid re-running test that are unrelated to e.g. a change of generic for another tb. Though if a config file generates a thousand test case configurations for single tb.*vhd. Either those 1000 test will have to be re-run, or it could be split into multiple config files. Or if licenses isn’t a problem implement some test sharding strategy?

Thanks for taking time to the review the issue! =) Once I have a way of sidestepping vunit compilation I can start looking into ghdl rules, unless, they already exist somewhere. And in extension vunit rules.

[solsjo]

@umarcor I haven't looked into it that much but I've read forum posts about perserving the bazel cache online in https://github.com/gitpod-io/gitpod

But I'm not certain that that would work in-between users. There are most probably other alternatives as well.

[solsjo]

@umarcor, I just wanted to notify you that I've created bazel rules for both ghdl and vunit and setup an example here:

https://github.com/solsjo/rules_ghdl https://github.com/solsjo/rules_vunit https://github.com/solsjo/ghdl-example

if you want to try it out :)

[nathanaelhuffman]

@solsjo I'm interested in building some vunit rules for buck2 and figured taking a look at these would be helpful, but the https://github.com/solsjo/rules_vunit seems to be gone now? Is there a copy that is accessible somewhere?

[solsjo]

Hi @nathanaelhuffman!

Missed the email notification, sorry. I made it private a while back, but can make it public again.

From what remember I might have tied it up somewhat to some ghdl rules that I also wrote. Those rules however, are somewhat convoluted as I at the time was new to the build chain for hdl code. Still, some way of building the vunit hdl code using a build system might be a good idea even if VUnit's compilation is very performant.

I'll let you know when it has been made public again.