flashrom icon indicating copy to clipboard operation
flashrom copied to clipboard

1.2 + master: test suite is crashing

Open kloczek opened this issue 4 years ago • 13 comments

+ cd flashrom-1.2
+ /usr/bin/meson test -C x86_64-redhat-linux-gnu --num-processes 48 --print-errorlogs
ninja: Entering directory `/home/tkloczko/rpmbuild/BUILD/flashrom-1.2/x86_64-redhat-linux-gnu'
ninja: no work to do.
1/1 cmocka test flashrom FAIL           0.01s (exit status 10)

The output from the failed tests:

1/1 cmocka test flashrom FAIL           0.01s (exit status 10)

--- command ---
06:24:11 /home/tkloczko/rpmbuild/BUILD/flashrom-1.2/x86_64-redhat-linux-gnu/tests/flashrom_unit_tests
--- stdout ---
[==========] Running 7 test(s).
[ RUN      ] address_to_bits_test_success
[       OK ] address_to_bits_test_success
[ RUN      ] bitcount_test_success
[       OK ] bitcount_test_success
[ RUN      ] minmax_test_success
[       OK ] minmax_test_success
[ RUN      ] strcat_realloc_test_success
[       OK ] strcat_realloc_test_success
[ RUN      ] tolower_string_test_success
[       OK ] tolower_string_test_success
[ RUN      ] reverse_byte_test_success
[       OK ] reverse_byte_test_success
[ RUN      ] reverse_bytes_test_success
[       OK ] reverse_bytes_test_success
[==========] 7 test(s) run.
[==========] Running 1 test(s).
[ RUN      ] flashbuses_to_text_test_success
[       OK ] flashbuses_to_text_test_success
[==========] 1 test(s) run.
[==========] Running 10 test(s).
[ RUN      ] spi_write_enable_test_success
[  FAILED  ] spi_write_enable_test_success
[ RUN      ] spi_write_disable_test_success
[  FAILED  ] spi_write_disable_test_success
[ RUN      ] probe_spi_rdid_test_success
[  FAILED  ] probe_spi_rdid_test_success
[ RUN      ] probe_spi_rdid4_test_success
[  FAILED  ] probe_spi_rdid4_test_success
[ RUN      ] probe_spi_rems_test_success
[  FAILED  ] probe_spi_rems_test_success
[ RUN      ] probe_spi_res1_test_success
[  FAILED  ] probe_spi_res1_test_success
[ RUN      ] probe_spi_res2_test_success
[  FAILED  ] probe_spi_res2_test_success
[ RUN      ] probe_spi_res3_test_success
[  FAILED  ] probe_spi_res3_test_success
[ RUN      ] probe_spi_at25f_test_success
[  FAILED  ] probe_spi_at25f_test_success
[ RUN      ] probe_spi_st95_test_success
[  FAILED  ] probe_spi_st95_test_success
[==========] 10 test(s) run.
--- stderr ---
[  PASSED  ] 7 test(s).
[  PASSED  ] 1 test(s).
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  PASSED  ] 0 test(s).
[  FAILED  ] 10 test(s), listed below:
[  FAILED  ] spi_write_enable_test_success
[  FAILED  ] spi_write_disable_test_success
[  FAILED  ] probe_spi_rdid_test_success
[  FAILED  ] probe_spi_rdid4_test_success
[  FAILED  ] probe_spi_rems_test_success
[  FAILED  ] probe_spi_res1_test_success
[  FAILED  ] probe_spi_res2_test_success
[  FAILED  ] probe_spi_res3_test_success
[  FAILED  ] probe_spi_at25f_test_success
[  FAILED  ] probe_spi_st95_test_success

 10 FAILED TEST(S)
-------


Summary of Failures:

1/1 cmocka test flashrom FAIL           0.01s (exit status 10)
1/1 cmocka test flashrom FAIL           0.01s (exit status 10)

Ok:                 0
Expected Fail:      0
Fail:               1
Unexpected Pass:    0
Skipped:            0
Timeout:            0

Despite messages about sigsegv I don't see any core files generated by this crash :/

kloczek avatar Jan 16 '21 06:01 kloczek

Source code setup:

%meson \
        -D config_atahpt=true \
        -D config_atapromise=true \
        -D config_atavia=true \
        -D config_buspirate_spi=true \
        -D config_ch341a_spi=true \
        -D config_dediprog=true \
        -D config_developerbox_spi=true \
        -D config_digilent_spi=true \
        -D config_drkaiser=true \
        -D config_dummy=true \
        -D config_ft2232_spi=true \
        -D config_gfxnvidia=true \
        -D config_internal_dmi=true \
        -D config_internal=true \
        -D config_it8212=true \
        -D config_jlink_spi=true \
        -D config_linux_mtd=true \
        -D config_linux_spi=true \
        -D config_mstarddc_spi=true \
        -D config_nic3com=true \
        -D config_nicintel_eeprom=true \
        -D config_nicintel_spi=true \
        -D config_nicintel=true \
        -D config_nicnatsemi=true \
        -D config_nicrealtek=true \
        -D config_ogp_spi=true \
        -D config_pickit2_spi=true \
        -D config_pony_spi=true \
        -D config_rayer_spi=true \
        -D config_satamv=true \
        -D config_satasii=true \
        -D config_serprog=true \
        -D config_stlinkv3_spi=true \
        -D config_usbblaster_spi=true \
        -D pciutils=true \
        -D usb=true \
        %{nil}

kloczek avatar Jan 16 '21 06:01 kloczek

BTW do you have any plans to make new release soon? :)

kloczek avatar Jan 16 '21 06:01 kloczek

This is likely because of the use of LTO and cmocka's use of --wrap not playing nicely together: https://bugzilla.redhat.com/show_bug.cgi?id=1693831

So either disable LTO when building flashrom or perhaps there is a way to disable it just for the tests via meson?

alexmurray avatar May 19 '21 07:05 alexmurray

So either disable LTO when building flashrom or perhaps there is a way to disable it just for the tests via meson?

So far I was not able to find sometning like that for only test suite. Do ypu know how to do that? :)

kloczek avatar May 19 '21 08:05 kloczek

Sorry, this bug tracker instance in github has been activated by mistake. Please mail [email protected] to reach the developers. Thank you.

hailfinger avatar May 19 '21 09:05 hailfinger

So either disable LTO when building flashrom or perhaps there is a way to disable it just for the tests via meson?

So far I was not able to find sometning like that for only test suite. Do ypu know how to do that? :)

No I tried for a few hours with various ways to add -flto=none to c_args / link_args in the tests/meson.build but to no avail so for now I am just disabling LTO for the entire flashrom package.

alexmurray avatar May 19 '21 13:05 alexmurray

Am 19.05.21 um 15:04 schrieb Alex Murray:

    So either disable LTO when building flashrom or perhaps there
    is a way to disable it just for the tests via meson?

So far I was not able to find sometning like that for only test suite.
Do ypu know how to do that? :)

No I tried for a few hours with various ways to add |-flto=none| to c_args / link_args in the tests/meson.build but to no avail so for now I am just disabling LTO for the entire flashrom package.

The meson build scripts have a few known bugs and an estimated 70-80% of the features of the classic Makefile based build. I am not aware of any regular testing of the meson build scripts. They are provided for convenience in cases where a meson based build is desired.

hailfinger avatar May 20 '21 20:05 hailfinger

Could you please try with --num-processes 1 ? I suspect Meson to have a bug when running many tests in parallel.

ManuelAtWork avatar Aug 06 '21 13:08 ManuelAtWork

Negative. It doesn't help but again it is only when LTO is used. Witout LTO everything is fine.

+ /usr/bin/meson test -C x86_64-redhat-linux-gnu --num-processes 1 --print-errorlogs
ninja: Entering directory `/home/tkloczko/rpmbuild/BUILD/flashrom-1.2/x86_64-redhat-linux-gnu'
ninja: no work to do.
1/1 cmocka test flashrom        FAIL            0.01s   exit status 10
>>> MALLOC_PERTURB_=53 /home/tkloczko/rpmbuild/BUILD/flashrom-1.2/x86_64-redhat-linux-gnu/tests/flashrom_unit_tests
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――― ✀  ―――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――
stdout:
[==========] Running 7 test(s).
[ RUN      ] address_to_bits_test_success
[       OK ] address_to_bits_test_success
[ RUN      ] bitcount_test_success
[       OK ] bitcount_test_success
[ RUN      ] minmax_test_success
[       OK ] minmax_test_success
[ RUN      ] strcat_realloc_test_success
[       OK ] strcat_realloc_test_success
[ RUN      ] tolower_string_test_success
[       OK ] tolower_string_test_success
[ RUN      ] reverse_byte_test_success
[       OK ] reverse_byte_test_success
[ RUN      ] reverse_bytes_test_success
[       OK ] reverse_bytes_test_success
[==========] 7 test(s) run.
[==========] Running 1 test(s).
[ RUN      ] flashbuses_to_text_test_success
[       OK ] flashbuses_to_text_test_success
[==========] 1 test(s) run.
[==========] Running 10 test(s).
[ RUN      ] spi_write_enable_test_success
[  FAILED  ] spi_write_enable_test_success
[ RUN      ] spi_write_disable_test_success
[  FAILED  ] spi_write_disable_test_success
[ RUN      ] probe_spi_rdid_test_success
[  FAILED  ] probe_spi_rdid_test_success
[ RUN      ] probe_spi_rdid4_test_success
[  FAILED  ] probe_spi_rdid4_test_success
[ RUN      ] probe_spi_rems_test_success
[  FAILED  ] probe_spi_rems_test_success
[ RUN      ] probe_spi_res1_test_success
[  FAILED  ] probe_spi_res1_test_success
[ RUN      ] probe_spi_res2_test_success
[  FAILED  ] probe_spi_res2_test_success
[ RUN      ] probe_spi_res3_test_success
[  FAILED  ] probe_spi_res3_test_success
[ RUN      ] probe_spi_at25f_test_success
[  FAILED  ] probe_spi_at25f_test_success
[ RUN      ] probe_spi_st95_test_success
[  FAILED  ] probe_spi_st95_test_success
[==========] 10 test(s) run.
stderr:
[  PASSED  ] 7 test(s).
[  PASSED  ] 1 test(s).
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  ERROR   ] --- Test failed with exception: Segmentation fault(11)
[  PASSED  ] 0 test(s).
[  FAILED  ] 10 test(s), listed below:
[  FAILED  ] spi_write_enable_test_success
[  FAILED  ] spi_write_disable_test_success
[  FAILED  ] probe_spi_rdid_test_success
[  FAILED  ] probe_spi_rdid4_test_success
[  FAILED  ] probe_spi_rems_test_success
[  FAILED  ] probe_spi_res1_test_success
[  FAILED  ] probe_spi_res2_test_success
[  FAILED  ] probe_spi_res3_test_success
[  FAILED  ] probe_spi_at25f_test_success
[  FAILED  ] probe_spi_st95_test_success

 10 FAILED TEST(S)
――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――――


Summary of Failures:

1/1 cmocka test flashrom FAIL            0.01s   exit status 10


Ok:                 0
Expected Fail:      0
Fail:               1
Unexpected Pass:    0
Skipped:            0
Timeout:            0

Full log written to /home/tkloczko/rpmbuild/BUILD/flashrom-1.2/x86_64-redhat-linux-gnu/meson-logs/testlog.txt

kloczek avatar Aug 06 '21 20:08 kloczek

Ok, thank's for trying!

ManuelAtWork avatar Aug 09 '21 11:08 ManuelAtWork

Please let me know if you may have some fix for that issue in form of exact commit or PR. I can in few seconds integrate build of any of my packages with addtional commit available in git repo. Thx.

kloczek avatar Aug 09 '21 11:08 kloczek

Sure! If we find a solution (or further clues), then we will let you know. :-) I'll stay subscribed to this ticket ...

ManuelAtWork avatar Aug 09 '21 11:08 ManuelAtWork

Any progress? :) BTW .. after +1.5y I think that it would be good to make new release :P

kloczek avatar Sep 19 '21 22:09 kloczek