thin-edge.io icon indicating copy to clipboard operation
thin-edge.io copied to clipboard

"Sporadic" test failures in c8y tests

Open matthiasbeyer opened this issue 1 year ago • 3 comments

Describe the bug

There's a "sporadic" test failure in mapper_publishes_software_update_request(). Unfortunately, the "sporadicness" of the test is that it failed two out of three times for PR #1283 - which is not sporadic at all.

Here's one of the runs of that PR: https://github.com/thin-edge/thin-edge.io/runs/7464719118?check_suite_focus=true#step:7:1243

This error should not happen.

I guess it happens because the in-process testing broker that is spun up for each test is not yet in a fully functional state but the test already tries to send data to it. Because of this, we get a connection reset.

OR it may be because some malformed MQTT paket is sent to the broker, as brokers disconnect on malformed pakets.

To Reproduce

Not sure.

matthiasbeyer avatar Jul 22 '22 09:07 matthiasbeyer

It's related to MQTT connection.

---- c8y::tests::mapper_publishes_software_update_request stdout ----
thread 'c8y::tests::mapper_publishes_software_update_request' panicked at 'called `Result::unwrap()` on an `Err` value: I/O: connection closed by peer
Caused by:
    connection closed by peer', crates/core/tedge_mapper/src/c8y/tests.rs:79:57

I'm not certainly sure, however, I feel sometimes GitHub hosted runner kills the connection and make tests failed.

For example, another sporadic failure I observed is also MQTT related, Connection Error I/O: Connection refused (os error 111). https://github.com/thin-edge/thin-edge.io/pull/1292#issuecomment-1193766699

rina23q avatar Jul 25 '22 09:07 rina23q

From GH Actions, looks like this is happening on armv7 architecture.

PradeepKiruvale avatar Aug 25 '22 10:08 PradeepKiruvale

Another failure:

---- c8y::tests::c8y_mapper_child_alarm_mapping_to_smartrest stdout ----
thread 'c8y::tests::c8y_mapper_child_alarm_mapping_to_smartrest' panicked at 'called `Result::unwrap()` on an `Err` value: I/O: connection closed by peer

Caused by:
    connection closed by peer', crates/core/tedge_mapper/src/c8y/tests.rs:406:10
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

rina23q avatar Sep 05 '22 11:09 rina23q

Resolved by #1567. It's the fix of test.

rina23q avatar Nov 09 '22 14:11 rina23q