thesofproject
cmocka: add eq_fir unit test
@singalsu @andrula-song this PR uses pass-through coef. Please help to figure out a non-pass-through coef case. Thanks!
@singalsu @andrula-song this PR uses pass-through coef. Please help to figure out a non-pass-through coef case. Thanks!
Thanks! I will add the Matlab scripts to generate the test vectors for pass-through and some other response shape, e.g. the Loudness EQ in examples for FIR. The the process to create the numbers blobs will be then documented in the scripts.
@RanderWang we have the infra for unifying the unit tests for modules using the module adapter now. Please have a look at volume for example. Could you please use that as well?
@RanderWang we have the infra for unifying the unit tests for modules using the module adapter now. Please have a look at volume for example. Could you please use that as well?
Sure, I will. The most important thing for me is to study FIR algorithm and how to make it work. We need to make a usable FIR test first then I can change interface or framework.
@RanderWang pls check CI
[ 52%] Building C object test/cmocka/src/audio/eq_fir/CMakeFiles/audio_for_eq_fir.dir/__/__/__/__/__/src/audio/eq_fir/eq_fir.c.o
/home/runner/work/sof/sof/src/audio/eq_fir/eq_fir.c:642:16: error: no previous prototype for ‘sys_comp_eq_fir_init’ [-Werror=missing-prototypes]
642 | UT_STATIC void sys_comp_eq_fir_init(void)
| ^~~~~~~~~~~~~~~~~~~~
[ 53%] Building C object test/cmocka/src/audio/eq_iir/CMakeFiles/audio_for_eq_iir.dir/__/__/__/__/__/src/ipc/ipc-common.c.o
cc1: all warnings being treated as errors
make[2]: *** [test/cmocka/src/audio/eq_fir/CMakeFiles/audio_for_eq_fir.dir/build.make:76: test/cmocka/src/audio/eq_fir/CMakeFiles/audio_for_eq_fir.dir/__/__/__/__/__/src/audio/eq_fir/eq_fir.c.o] Error 1
make[1]: *** [CMakeFiles/Makefile2:1623: test/cmocka/src/audio/eq_fir/CMakeFiles/audio_for_eq_fir.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
Even more important: check why a cmocka PR fails to compile cmocka in CI but not for you. If you use cmocka differently from CI then at least one of you is using cmocka wrong.
@lgirdwood it depends on https://github.com/thesofproject/sof/pull/6194 and 6194 should be merged first
@lgirdwood it depends on #6194 and 6194 should be merged first
Now merged.
@softwarecki Unit test is failed for this PR but it passed with ./script/run-cmock.sh locally. I am sure there is no missed change in local environment. Do you know how does CI test unit test ? thanks!
@RanderWang see the logs (missing Kconfig option or file in mock build ?)
https://sof-ci.01.org/sof-pr-viewer/#/build/PR6308/build10653042
[ 98%] Linking C executable eq_iir_process
cd /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/audio/eq_iir && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/eq_iir_process.dir/link.txt --verbose=1
/localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-xcc CMakeFiles/eq_iir_process.dir/eq_iir_process.c.o -o eq_iir_process ../../../libcommon_mock.a -T/quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/memory_mock.x /localdisk/tools/cmocka/build_cavs2x_LX6HiFi3_2017_8/install/lib/libcmocka-static.a ../../../libcommon_mock.a libaudio_for_eq_iir.a ../../../libcommon_mock.a -lm -lgcc -Wl,--no-check-sections -ucall_user_start -Wl,-static
make[2]: Leaving directory '/quickbuild/workspace1/24733/SOF_FW/build_ut'
[ 98%] Built target eq_iir_process
[100%] Linking C executable eq_fir_process
cd /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/audio/eq_fir && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/eq_fir_process.dir/link.txt --verbose=1
/localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-xcc CMakeFiles/eq_fir_process.dir/eq_fir_process.c.o -o eq_fir_process ../../../libcommon_mock.a -T/quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/memory_mock.x /localdisk/tools/cmocka/build_cavs2x_LX6HiFi3_2017_8/install/lib/libcmocka-static.a ../../../libcommon_mock.a libaudio_for_eq_fir.a ../../../libcommon_mock.a -lm -lgcc -Wl,--no-check-sections -ucall_user_start -Wl,-static
libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_free_delaylines':
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:167: undefined reference to `fir_delay_size'
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:167: undefined reference to `fir_init_coef'
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:167: undefined reference to `fir_reset'
libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_init_coef':
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:203: undefined reference to `fir_delay_size'
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:214: undefined reference to `fir_init_coef'
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:221: undefined reference to `fir_reset'
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:190: undefined reference to `fir_init_delay'
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:194: undefined reference to `fir_init_delay'
libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_init':
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:315: undefined reference to `fir_reset'
libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_set_alignment':
/quickbuild/workspace1/24733/SOF_FW/src/include/sof/audio/format.h:170: undefined reference to `eq_fir_2x_s32'
libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_set_alignment':
/quickbuild/workspace1/24733/SOF_FW/src/include/sof/audio/audio_stream.h:231: undefined reference to `eq_fir_2x_s16'
/quickbuild/workspace1/24733/SOF_FW/src/include/sof/audio/audio_stream.h:234: undefined reference to `eq_fir_2x_s24'
libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_prepare':
/quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:111: undefined reference to `fir_reset'
test/cmocka/src/audio/eq_fir/CMakeFiles/eq_fir_process.dir/build.make:102: recipe for target 'test/cmocka/src/audio/eq_fir/eq_fir_process' failed
make[2]: *** [test/cmocka/src/audio/eq_fir/eq_fir_process] Error 2
make[2]: Leaving directory '/quickbuild/workspace1/24733/SOF_FW/build_ut'
CMakeFiles/Makefile2:2625: recipe for target 'test/cmocka/src/audio/eq_fir/CMakeFiles/eq_fir_process.dir/all' failed
make[1]: *** [test/cmocka/src/audio/eq_fir/CMakeFiles/eq_fir_process.dir/all] Error 2
make[1]: *** Waiting for unfinished jobs....
[100%] Linking C executable pipeline_free
@RanderWang see the logs (missing Kconfig option or file in mock build ?)
[ 98%] Linking C executable eq_iir_process cd /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/audio/eq_iir && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/eq_iir_process.dir/link.txt --verbose=1 /localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-xcc CMakeFiles/eq_iir_process.dir/eq_iir_process.c.o -o eq_iir_process ../../../libcommon_mock.a -T/quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/memory_mock.x /localdisk/tools/cmocka/build_cavs2x_LX6HiFi3_2017_8/install/lib/libcmocka-static.a ../../../libcommon_mock.a libaudio_for_eq_iir.a ../../../libcommon_mock.a -lm -lgcc -Wl,--no-check-sections -ucall_user_start -Wl,-static make[2]: Leaving directory '/quickbuild/workspace1/24733/SOF_FW/build_ut' [ 98%] Built target eq_iir_process [100%] Linking C executable eq_fir_process cd /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/audio/eq_fir && /usr/local/bin/cmake -E cmake_link_script CMakeFiles/eq_fir_process.dir/link.txt --verbose=1 /localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-xcc CMakeFiles/eq_fir_process.dir/eq_fir_process.c.o -o eq_fir_process ../../../libcommon_mock.a -T/quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/memory_mock.x /localdisk/tools/cmocka/build_cavs2x_LX6HiFi3_2017_8/install/lib/libcmocka-static.a ../../../libcommon_mock.a libaudio_for_eq_fir.a ../../../libcommon_mock.a -lm -lgcc -Wl,--no-check-sections -ucall_user_start -Wl,-static libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_free_delaylines': /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:167: undefined reference to `fir_delay_size' /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:167: undefined reference to `fir_init_coef' /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:167: undefined reference to `fir_reset' libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_init_coef': /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:203: undefined reference to `fir_delay_size' /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:214: undefined reference to `fir_init_coef' /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:221: undefined reference to `fir_reset' /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:190: undefined reference to `fir_init_delay' /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:194: undefined reference to `fir_init_delay' libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_init': /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:315: undefined reference to `fir_reset' libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_set_alignment': /quickbuild/workspace1/24733/SOF_FW/src/include/sof/audio/format.h:170: undefined reference to `eq_fir_2x_s32' libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_set_alignment': /quickbuild/workspace1/24733/SOF_FW/src/include/sof/audio/audio_stream.h:231: undefined reference to `eq_fir_2x_s16' /quickbuild/workspace1/24733/SOF_FW/src/include/sof/audio/audio_stream.h:234: undefined reference to `eq_fir_2x_s24' libaudio_for_eq_fir.a(eq_fir.c.o): In function `eq_fir_prepare': /quickbuild/workspace1/24733/SOF_FW/src/audio/eq_fir/eq_fir.c:111: undefined reference to `fir_reset' test/cmocka/src/audio/eq_fir/CMakeFiles/eq_fir_process.dir/build.make:102: recipe for target 'test/cmocka/src/audio/eq_fir/eq_fir_process' failed make[2]: *** [test/cmocka/src/audio/eq_fir/eq_fir_process] Error 2 make[2]: Leaving directory '/quickbuild/workspace1/24733/SOF_FW/build_ut' CMakeFiles/Makefile2:2625: recipe for target 'test/cmocka/src/audio/eq_fir/CMakeFiles/eq_fir_process.dir/all' failed make[1]: *** [test/cmocka/src/audio/eq_fir/CMakeFiles/eq_fir_process.dir/all] Error 2 make[1]: *** Waiting for unfinished jobs.... [100%] Linking C executable pipeline_free
@lgirdwood I can't reproduce it with my environment. All these symbols are included and be built. So I want to know how does CI build it
A while back I discovered QB uses a "static" version of CMocka:
- https://github.com/thesofproject/sof-docs/pull/395#issuecomment-1005267907
Github Actions builds the same commit fine: https://github.com/thesofproject/sof/actions/runs/3216683629/jobs/5258822444
A while back I discovered QB uses a "static" version of CMocka:
Github Actions builds the same commit fine: https://github.com/thesofproject/sof/actions/runs/3216683629/jobs/5258822444
Thanks, Marc, so it is caused by staic version of CMocka ? @kkarask
@lgirdwood the error in unit test is not caused by my PR. please check marc's comment. This PR works with local test. The error is caused by static version of CMocka
Github Actions builds the same commit fine: https://github.com/thesofproject/sof/actions/runs/3216683629/jobs/5258822444
@lgirdwood the error in unit test is not caused by my PR. please check marc's comment. This PR works with local test. The error is caused by static version of CMocka
Github Actions builds the same commit fine: https://github.com/thesofproject/sof/actions/runs/3216683629/jobs/5258822444
Ok, I think we have a GH using x86 cmocka with GCC and internal CI using XCC and xtensa target. @RanderWang can you make the update so XCC cmocka builds work.
please check marc's comment. This PR works with local test. The error is caused by static version of CMocka
I did not say that. I mentioned one difference I found but the root cause could be something else.
It looks like the tests are now compiling in https://sof-ci.01.org/sof-pr-viewer/#/build/PR6308/build10629900 but they are not all passing.
@singalsu now We get a few samples error in only S24_LE test with XCC built FW, no such issue with GCC. The ref = 0x7FFFFF, but FW output = 0xFF800000. @andrula-song will check it. Thanks!
@singalsu now We get a few samples error in only S24_LE test with XCC built FW, no such issue with GCC. The ref = 0x7FFFFF, but FW output = 0xFF800000. @andrula-song will check it. Thanks!
here is the fix: https://github.com/thesofproject/sof/pull/6468
@RanderWang still some errors in CI UT. @singalsu @andrula-song @ShriramShastry can you guys help here to advise on how to fix.
56/58 Testing: matrix
56/58 Test: matrix
Command: "/localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-run" "--exit_with_target_code" "matrix"
Directory: /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/math/matrix
"matrix" start time: Oct 24 13:10 CEST
Output:
----------------------------------------------------------
1..4
Max absolute error = 0.00 (max 1.50), error RMS = 0.00 (max 0.50)
ok 1 - test_matrix_mult_16_test1
Max absolute error = 0.00 (max 1.50), error RMS = 0.00 (max 0.50)
ok 2 - test_matrix_mult_16_test2
Max absolute error = 0.00 (max 1.50), error RMS = 0.00 (max 0.50)
ok 3 - test_matrix_mult_16_test3
Max absolute error = 0.00 (max 1.50), error RMS = 0.00 (max 0.50)
ok 4 - test_matrix_mult_16_test4
# ok - tests
<end of output>
Test time = 5.51 sec
----------------------------------------------------------
Test Passed.
"matrix" end time: Oct 24 13:10 CEST
"matrix" time elapsed: 00:00:05
----------------------------------------------------------
55/58 Testing: window
55/58 Test: window
Command: "/localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-run" "--exit_with_target_code" "window"
Directory: /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/math/window
"window" start time: Oct 24 13:10 CEST
Output:
----------------------------------------------------------
1..4
Window rectangular RMS error = 0.00 LSB (max 1.00)
ok 1 - test_math_window_rectangular
Window blackman RMS error = 0.67 LSB (max 1.00)
ok 2 - test_math_window_blackman
Window hamming RMS error = 0.00 LSB (max 1.00)
ok 3 - test_math_window_hamming
Window povey RMS error = 0.29 LSB (max 1.00)
ok 4 - test_math_window_povey
# ok - tests
<end of output>
Test time = 6.15 sec
----------------------------------------------------------
Test Passed.
"window" end time: Oct 24 13:10 CEST
"window" time elapsed: 00:00:06
----------------------------------------------------------
57/58 Testing: auditory
57/58 Test: auditory
Command: "/localdisk/tools/xtensa/RG-2017.8-linux/XtensaTools/bin/xt-run" "--exit_with_target_code" "auditory"
Directory: /quickbuild/workspace1/24733/SOF_FW/build_ut/test/cmocka/src/math/auditory
"auditory" start time: Oct 24 13:10 CEST
Output:
----------------------------------------------------------
1..10
Max absolute error = 3.00 (max 5.00), error RMS = 0.91 (max 3.00)
ok 1 - test_mel_filterbank_16_test1
Max absolute error = 3.00 (max 5.00), error RMS = 0.88 (max 3.00)
ok 2 - test_mel_filterbank_32_test1
Max absolute error = 1.00 (max 5.00), error RMS = 0.59 (max 3.00)
ok 3 - test_mel_filterbank_16_test2
Max absolute error = 1.00 (max 5.00), error RMS = 0.47 (max 3.00)
ok 4 - test_mel_filterbank_32_test2
Max absolute error = 1.00 (max 5.00), error RMS = 0.55 (max 3.00)
ok 5 - test_mel_filterbank_16_test3
Max absolute error = 1.00 (max 5.00), error RMS = 0.45 (max 3.00)
ok 6 - test_mel_filterbank_32_test3
Max absolute error = 2.00 (max 5.00), error RMS = 1.04 (max 3.00)
ok 7 - test_mel_filterbank_16_test4
Max absolute error = 2.00 (max 5.00), error RMS = 1.25 (max 3.00)
ok 8 - test_mel_filterbank_32_test4
Max absolute error = 1.00 (max 1.50), error RMS = 0.22 (max 0.50)
ok 9 - test_hz_to_mel
Max absolute error = 3.00 (max 5.00), error RMS = 1.30 (max 1.50)
ok 10 - test_mel_to_hz
# ok - tests
<end of output>
Test time = 6.90 sec
------
error: 17: not ok 2 - test_audio_eq_fir 17: # 0xffffffffff800000 != 0x7fffff 17: # /quickbuild/workspace1/24733/SOF_FW/test/cmocka/src/audio/eq_fir/eq_fir_process.c:324: error: Failure!
The last error was : The ref = 0x7FFFFF, but FW output = 0xFF800000 before https://github.com/thesofproject/sof/pull/6468 @singalsu @andrula-song do you have any advice ?
error: 17: not ok 2 - test_audio_eq_fir 17: # 0xffffffffff800000 != 0x7fffff 17: # /quickbuild/workspace1/24733/SOF_FW/test/cmocka/src/audio/eq_fir/eq_fir_process.c:324: error: Failure!
Looks like we are comparing a 24bit number to a 64bit number - > do we need a conversion for xt-run or host ?
The last error was : The ref = 0x7FFFFF, but FW output = 0xFF800000 before #6468 @singalsu @andrula-song do you have any advice ?
SOFCI TEST
Only sof-ci/jenkins knows these magic words and it does not run CMocka. You must force-push. Pro tip: git commit --amend -C HEAD
SOFCI TEST
Only sof-ci/jenkins knows these magic words and it does not run CMocka. You must force-push. Pro tip:
git commit --amend -C HEAD
Thanks! @lgirdwood pass eq_fir now. need to trigger cmoka test again