Rust bindings tests in the s2n-tls-tokio crate are flaky on macOS

[goatgoose]

Security issue notifications

If you discover a potential security issue in s2n we ask that you notify AWS Security via our vulnerability reporting page. Please do not create a public github issue.

Problem:

When running cargo test inside the s2n-tls-tokio crate on macOS, sometimes the tests fail. The failing tests are the following:

• tests/handshake.rs: handshake_error_with_blinding
• tests/shutdown.rs: shutdown_after_split

Sometimes all of the tests pass with no failures. The following is an example of a failure:

     Running tests/handshake.rs (/Users/vclarksa/w/s2n-tls-fork/bindings/rust/target/debug/deps/handshake-0c56a7123ca72181)

running 6 tests
test handshake_error ... ok
test handshake_basic ... ok
test handshake_error_with_blinding ... FAILED
test handshake_with_connection_config ... ok
test handshake_with_connection_config_with_pool ... ok
test handshake_with_pool_multithread ... ok

failures:

---- handshake_error_with_blinding stdout ----
Error: Elapsed(())
thread 'handshake_error_with_blinding' panicked at 'assertion failed: `(left == right)`
  left: `1`,
 right: `0`: the test returned a termination value with a non-zero status code (1) which indicates a failure', /rustc/f1edd0429582dd29cccacaf50fd134b05593bd9c/library/test/src/lib.rs:195:5
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace


failures:
    handshake_error_with_blinding

test result: FAILED. 5 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out; finished in 0.13s

error: test failed, to rerun pass '--test handshake'

Solution:

A description of the possible solution in terms of S2N architecture. Highlight and explain any potentially controversial design decisions taken.

• Does this change what S2N sends over the wire? If yes, explain. N/A
• Does this change any public APIs? If yes, explain. N/A
• Which versions of TLS will this impact? N/A

Requirements / Acceptance Criteria:

What must a solution address in order to solve the problem? How do we know the solution is complete?

• RFC links: Links to relevant RFC(s)
• Related Issues: Link any relevant issues
• Will the Usage Guide or other documentation need to be updated?
• Testing: How will this change be tested? Call out new integration tests, functional tests, or particularly interesting/important unit tests.
  • Will this change trigger SAW changes? Changes to the state machine, the s2n_handshake_io code that controls state transitions, the DRBG, or the corking/uncorking logic could trigger SAW failures.
  • Should this change be fuzz tested? Will it handle untrusted input? Create a separate issue to track the fuzzing work.

Out of scope:

Is there anything the solution will intentionally NOT address?