zeek
Integrate CI with Coverity
We should see if we can integrate the runtime with Coverity, and maybe some generated code as well.
Less important for the compiler.
As a payload for coverity to analyze we could pass it the sources of the runtime libraries, and an example JIT output exercising some of the functionality.
Since coverity requires not openly accessible wrapper tools, I am unsure how this can be implemented completely open.
As an exploration, I added their tools to our ubuntu-20 Docker container. Neither with Clang nor GCC was I able to analyze the required 85% of source files (I got around 65% when building all of Spicy, and around 50% when only building hilti-rt).
IMPORTANT - Your build will be rejected if at least 85% units of code are not compiled.
tail cov-int/build-log.txt The last few lines of cov-int/build-log.txt should look as follows to indicate that at least 85% of the compilation units are compiled successfully to avoid false-positives:Compilation units (85%) are ready for analysis The cov-build utility completed successfully
$ tail /opt/spicy/src/cov-int/build-log.txt
2020-12-16T15:42:21.422185Z|cov-build|764|info|>
2020-12-16T15:42:21.424191Z|cov-build|764|info|> Build time (C/C++ emits total): 00:25:19.225282
2020-12-16T15:42:21.424191Z|cov-build|764|info|>
2020-12-16T15:42:21.424191Z|cov-build|764|info|>
2020-12-16T15:42:21.424227Z|cov-build|764|info|> [WARNING] Emitted 160 C/C++ compilation units (65%) successfully
2020-12-16T15:42:21.424227Z|cov-build|764|info|> [WARNING] Recoverable errors were encountered during 129 of these C/C++ compilation units.
2020-12-16T15:42:21.424227Z|cov-build|764|info|>
2020-12-16T15:42:21.424227Z|cov-build|764|info|> 160 C/C++ compilation units (65%) are ready for analysis
2020-12-16T15:42:21.424227Z|cov-build|764|info|> For more details, please look at:
2020-12-16T15:42:21.424227Z|cov-build|764|info|> /opt/spicy/src/cov-int/build-log.txt
Looking at the logs, many of the issues seem to be around C++17 constructs like optional or variant. I wasn't able to find a definite answer on what version of their product is included in their open source offering, but it seems other projects have issues with C++17 support as well. The latest product-related update I found was https://community.synopsys.com/s/article/Does-Coverity-Analysis-support-C-17, but people still reported issued after that release was published.
The way currently look I think we are blocked by Coverity support for C++17 and should come back to this later. With that I feel we should remove the 1.0 milestone. WDYT @rsmmr?