Unit tests fail on certain platforms

[Jawfish]

After #504 was merged, running python -m unittest on an untouched clone of the repo has failures on Windows and Ubuntu, but not MacOS.

Windows 11:

FAIL: test_chord (tests.test_combos.TestCombo)
AssertionError: reports don't match up
FAIL: test_sequence (tests.test_combos.TestCombo)
AssertionError: reports don't match up
FAIL: test_holdtap_chain (tests.test_hold_tap.TestHoldTap)
AssertionError: reports don't match up
FAIL: test_oneshot (tests.test_hold_tap.TestHoldTap)
AssertionError: reports don't match up
FAIL: test_modtap (tests.test_tapdance.TestTapDance)
AssertionError: reports don't match up
FAIL: test_normal_key (tests.test_tapdance.TestTapDance)
AssertionError: reports don't match up

Ubuntu 20.04:

FAIL: test_chord (tests.test_combos.TestCombo)
AssertionError: reports don't match up

However, make test passes.

[xs5871]

Now it fails in the runner again. Can not reproduce with python3.10. Can reproduce with python3.8.

[Jawfish]

I get failures the following cases: Python 3.10.5 on Windows 11 Python 3.8.10 on Ubuntu 20.04, same machine as Windows 11 Python 3.10.5, on Arch, different machine I don't get any failures on Python 3.9.10 on macOS.

By removing the sleep code from keyboard_test.py, I can pass consistently* on Windows and Arch: Before

def do_main_loop(self):
    self.keyboard._main_loop()
    time.sleep(0.001)
    self.keyboard._main_loop()
    time.sleep(0.001)

After

def do_main_loop(self):
    self.keyboard._main_loop()

*I got a fail for test_chord the first time running python -m unittest on each platform, but not thereafter. It continues to fail on Ubuntu 20.04.

I'm not familiar enough with this class to determine if that's an appropriate solution, but it seems the issue is related to how various machines/operating systems track time, and maybe some differences between how it's handled with different Python versions under the hood.

[xs5871]

I put the sleep statement in to emulate KMKs maximum loop execution frequency. I get the tests to pass if I paste a bunch of self.keyboard._main_loop() at the end of do_main_loop.

[kdb424]

Can this be closed now, or is this still an issue?

[Jawfish]

It still fails on native Windows.

[LukeDRussell]

This seems to be resolved under PR571. I guess the failures have something to do with the somewhat out-of-date dependencies.

[xs5871]

Resolved by #571