sbt test failed

[vijayank88]

Type of issue: bug report

Impact: no functional change | API addition (no impact on existing code) | API modification | unknown

Development Phase: request | proposal

If the current behavior is a bug, please provide the steps to reproduce the problem: I've followed below steps and got stuck at sbt test.

git clone https://github.com/chipsalliance/chisel3.git
cd chisel3
sbt compile
sbt test
Got failed 1 test and log shown below:
g++  -I.  -MMD -I/usr/share/verilator/include -I/usr/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=1 -faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -Wno-undefined-bool-conversion -O1 -DTOP_TYPE=VModuleIODynamicIndexTester -DVL_USER_FINISH -include VModuleIODynamicIndexTester.h   -c -o VModuleIODynamicIndexTester__ALLsup.o VModuleIODynamicIndexTester__ALLsup.cpp
ar -cr VModuleIODynamicIndexTester__ALL.a VModuleIODynamicIndexTester__ALLcls.o VModuleIODynamicIndexTester__ALLsup.o
ranlib VModuleIODynamicIndexTester__ALL.a
g++    top.o verilated.o verilated_vcd_c.o VModuleIODynamicIndexTester__ALL.a    -o VModuleIODynamicIndexTester -lm -lstdc++ 
make: Leaving directory '/home/vijayan/CHISEL_THREE/chisel3/test_run_dir/ModuleIODynamicIndexTester/202110201626225768060872132690812'
[info] - Dynamic indexing of a Vec of Module IOs should work
[info] - It should be possible to bulk connect a Vec and a Seq
[error] Vec.scala:435: Vec and Seq being bulk connected have different lengths! in class chiselTests.VecSpec$$anon$10
[error] There were 1 error(s) during hardware elaboration.
[info] - Bulk connecting a Vec and Seq of different sizes should report a ChiselException
[info] - It should be possible to initialize a Vec with DontCare
[info] - Indexing a Chisel type Vec by a hardware type should give a sane error message
[info] Run completed in 48 minutes, 57 seconds.
[info] Total number of tests run: 955
[info] Suites: completed 149, aborted 0
[info] Tests: succeeded 954, failed 1, canceled 0, ignored 27, pending 0
[info] *** 1 TEST FAILED ***
[error] Failed tests:
[error] 	chiselTests.TraceSpec
[error] (Test / test) sbt.TestsFailedException: Tests unsuccessful
[error] Total time: 2943 s (49:03), completed 20-Oct-2021, 4:26:28 PM

What is the current behavior? 1 sbt test got failed and unsuccessful.

What is the expected behavior? sbt test to be successful for all tests.

Please tell us about your environment: OS: ubuntu 20.04

[vijayank88]

If I use below command all tests got successful. sbt

noPluginTests / test

Logs: [info] MissingCloneBindingExceptionSpec: [info] missing cloneType in Chisel3 [info] missing cloneType in Chisel2 [info] Run completed in 7 seconds, 556 milliseconds. [info] Total number of tests run: 43 [info] Suites: completed 8, aborted 0 [info] Tests: succeeded 43, failed 0, canceled 0, ignored 0, pending 0 [info] All tests passed. [success] Total time: 39 s, completed 20-Oct-2021, 4:52:59 PM

[jeehoonkang]

Just for the record, I'm experiencing the same error (line number being different):

[error] Vec.scala:465: Vec and Seq being bulk connected have different lengths! in class chiselTests.VecSpec$$anon$10

[jackkoenig]

Thank you @jeehoonkang for bringing this issue back to my attention.

For @vijayank88, I have run into the same issue and it is solved by using a newer version of verilator, TraceSpec uses an argument to verilator that is relatively new. I'm using verilator 4.200 on my Mac and verilator 4.204 on my Linux machine.

@jeehoonkang, that message looks scary but it's actually a negative test, the test is expecting and checking for failure, so this is actually not a failure. Taken from my own run of the tests:

[error] Vec.scala:465: Vec and Seq being bulk connected have different lengths! in class chiselTests.VecSpec$$anon$10                                                                                             
[error] There were 1 error(s) during hardware elaboration.
[info] - Bulk connecting a Vec and Seq of different sizes should report a ChiselException
[info] - It should be possible to initialize a Vec with DontCare
[info] - Indexing a Chisel type Vec by a hardware type should give a sane error message
make: Entering directory '/scratch/koenig/chisel3-backports/test_run_dir/ReduceTreeTester/20211222111203180566332233708383'                                                                                       
ccache g++  -I.  -MMD -I/sifive/tools/verilator/4.204-redhat/share/verilator/include -I/sifive/tools/verilator/4.204-redhat/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=1 -DVM_TRACE_FST=0
-faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -Wno-undefined-bool-conversion
-O1 -DTOP_TYPE=VReduceTreeTester -DVL_USER_FINISH -include VReduceTreeTester.h  -std=gnu++14 -Os -c -o top.o /scratch/koenig/chisel3-backports/test_run_dir/ReduceTreeTester/20211222111203180566332233708383/top.cpp
ccache g++  -I.  -MMD -I/sifive/tools/verilator/4.204-redhat/share/verilator/include -I/sifive/tools/verilator/4.204-redhat/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=1 -DVM_TRACE_FST=0
-faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -Wno-undefined-bool-conversion
-O1 -DTOP_TYPE=VReduceTreeTester -DVL_USER_FINISH -include VReduceTreeTester.h  -std=gnu++14 -Os -c -o verilated.o /sifive/tools/verilator/4.204-redhat/share/verilator/include/verilated.cpp                     
ccache g++  -I.  -MMD -I/sifive/tools/verilator/4.204-redhat/share/verilator/include -I/sifive/tools/verilator/4.204-redhat/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=1 -DVM_TRACE_FST=0
-faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -Wno-undefined-bool-conversion
-O1 -DTOP_TYPE=VReduceTreeTester -DVL_USER_FINISH -include VReduceTreeTester.h  -std=gnu++14 -Os -c -o verilated_vcd_c.o /sifive/tools/verilator/4.204-redhat/share/verilator/include/verilated_vcd_c.cpp         
/usr/bin/perl /sifive/tools/verilator/4.204-redhat/share/verilator/bin/verilator_includer -DVL_INCLUDE_OPT=include VReduceTreeTester.cpp VReduceTreeTester__Trace.cpp VReduceTreeTester__Slow.cpp VReduceTreeTester__Syms.cpp VReduceTreeTester__Trace__Slow.cpp > VReduceTreeTester__ALL.cpp
echo "" > VReduceTreeTester__ALL.verilator_deplist.tmp
ccache g++  -I.  -MMD -I/sifive/tools/verilator/4.204-redhat/share/verilator/include -I/sifive/tools/verilator/4.204-redhat/share/verilator/include/vltstd -DVM_COVERAGE=0 -DVM_SC=0 -DVM_TRACE=1 -DVM_TRACE_FST=0
-faligned-new -fcf-protection=none -Wno-bool-operation -Wno-sign-compare -Wno-uninitialized -Wno-unused-but-set-variable -Wno-unused-parameter -Wno-unused-variable -Wno-shadow     -Wno-undefined-bool-conversion
-O1 -DTOP_TYPE=VReduceTreeTester -DVL_USER_FINISH -include VReduceTreeTester.h  -std=gnu++14 -Os -c -o VReduceTreeTester__ALL.o VReduceTreeTester__ALL.cpp                                                        
Archive ar -rcs VReduceTreeTester__ALL.a VReduceTreeTester__ALL.o
g++    top.o verilated.o verilated_vcd_c.o VReduceTreeTester__ALL.a      -o VReduceTreeTester
rm VReduceTreeTester__ALL.verilator_deplist.tmp
make: Leaving directory '/scratch/koenig/chisel3-backports/test_run_dir/ReduceTreeTester/20211222111203180566332233708383'                                                                                        
[info] - reduceTree should preserve input/output type
[info] Run completed in 2 minutes, 35 seconds.
[info] Total number of tests run: 997
[info] Suites: completed 151, aborted 0
[info] Tests: succeeded 997, failed 0, canceled 0, ignored 27, pending 0
[info] All tests passed.
[success] Total time: 179 s (02:59), completed Dec 22, 2021, 11:12:04 AM

You can see the [error], but the bottom says "All tests passed" which is the actual result of sbt test.

As a tip, whenever a specific test fails like it did for @vijayank88 above, you can run just that one specific test with:

$ sbt
> testOnly chiselTests.TraceSpec

[jackkoenig]

@sequencer the TraceSpec is relying on newer verilator than many get with their package install. Do you think you could take a look at making it work with older verilator? Ideally 4.028 (Ubuntu 20.04 and Fedora 32). I believe it is the --build argument here: https://github.com/chipsalliance/chisel3/blob/72d5be9634c677e47ec8fce204be55c8936166fb/src/test/scala/chiselTests/experimental/TraceSpec.scala#L158

I had the same issue as @vijayank88 when doing the 3.5.0-RC2 release and had to bump.

[sequencer]

OK, I'll try to fix it. (I think we should catch it in our CI as well.)