trezor-firmware
trezor-firmware copied to clipboard
Model R UI tests
Connected with https://github.com/trezor/trezor-firmware/issues/2246:
- making sure the
UItests can be run for modelR
A lot of layouts had to be mocked just to be able to run all the tests, so they show at least something on the screen and do not raise NotImplementedError - see core/src/trezor/ui/layouts/tr/__init__.py.
Local results of the whole device_test suite is 258 failed, 736 passed, 90 skipped, 7 xfailed, 6 errors.
Next step is to go test-by-test and try to make it work while improving the visual side of it.
First possibility of looking at the UI test screens in CI - https://satoshilabs.gitlab.io/-/trezor/trezor-firmware/-/jobs/2416495841/artifacts/test_ui_report/index.html
Currently, my local results are 81 failed, 913 passed, 96 skipped, 7 xfailed.
The biggest part of the failures is from reset_recovery because these things are not yet implemented for R.
The second group of failures are tests asserting something in layout.lines, and they fail because we are not showing the correct things on the screen.
Some tests needed an explicit failure for model R with something like
if client.features.model == "R":
pytest.fail("causes freeze on the pagination screen")
because these cases would freeze the whole test pipeline. I put skip_tr only on the same spots as skip_t2 was (apart from sd_card functionality which will not ever be there), so we see what functionality is missing.
I would say the setup-UI-tests step is almost ready now and the next phase is make-UI-tests-pretty. @matejcik should I attempt to do something here, or wait for UI/UX people to give me concrete design for every screen?
we probably need to add "allow fail" to the R test CI job for now? (that said the current failure looks completely wrong)
we probably need to add "allow fail" to the R test CI job for now? (that said the current failure looks completely wrong)
Allow fail added in f048542. It should also fix the failing tests/device_tests/bitcoin/test_signtx_replacement.py::test_p2pkh_fee_bump, I have merged it incorrectly before
fwiw I'm working on fixing the UTF issue -- it's not that Rust can't deal with UTF, but the text passed in is just right to trigger a char boundary bug in our code
Force-pushed after rebase on master and to be compatible with https://github.com/trezor/trezor-firmware/pull/2277, which build on top of this
Replaced by https://github.com/trezor/trezor-firmware/pull/2610