FOEDAG
FOEDAG copied to clipboard
os-fpga
Initial TCL testing framework prototype
Note: This PR isn't intended for merging.
This is to show a tcl/qt testing prototype. The goal is to show an initial idea on how we might automate our gui testing and seek feedback and possible issues etc.
More details on the implementation, future goals, and current issue can be found under [RG-144]
There are 3 basic examples under tests/TestGui/Automated to show what will be involved to automate our gui. Anything that uses a modal dialog requires some rather messy logic currently.
I've added a proof of concept way of manipulating the ui after a modal dialog starts blocking. This can be demonstrated in the console with qt_dlg_test. Next step is to come up with a queue of some sort where we can collect commands we'll fire while the modal dialog is blocking.
Looks good to me. How to know that test passed? How to get actual and expected values and compare them? could you please provide something?
How to know that test passed? How to get actual and expected values and compare them? could you please provide something?
Those haven't been developed yet, but they fall under the "Verification Helpers" and "Widget Introspection" bullet points on the confluence page's implementation status.
For Verification Helpers, we just need to write a couple helper functions that will error out on common checks like checking a log file for text, or asserting that a value is as expected. A basic example of failing out of a tcl test can be found here: https://github.com/os-fpga/FOEDAG/blob/6f3def06791c367f962396d1036390c93b62c50f/tests/TestBatch/test_ip_generate.tcl#L45
For Widget Introspection, we need to build out more introspection functions in the tcl interface. Currently there is qt_isEnabled which takes a widget pointer string and then calls isEnabled() on it and returns the result. The most common example will probably be getText() which would work the same where you pass a pointer, but we'll need handlers for different widget types as not all widgets support the text() function etc. Once we have functions that can check live gui values like that, then the test writer can check and error out on whatever value they care about.
Added example tests to tests/TestGui/Automated. One demonstrates how to deal w/ a modal dialog, then other explores how we currently need to try/catch our scripts to catch errors when running w/ --script.
