solar
solar copied to clipboard
satoshipay
E2E testing
Add at least a few simple end-to-end tests. Use spectron to check the electron app.
See https://dzone.com/articles/write-automated-tests-for-electron-with-spectron-m.
Investigate whether taking snapshots (https://github.com/electron/spectron#capturepage) and testing with shutter.sh makes sense.
Should be top priority. You are merging bugfixes (bugfixes as in "I hope this fixes the bug which I was not able to reproduce at all but I am pushing some code changes anyhow just in case...") without regression tests. Very dangerous...
Well, the issue with this is two-fold. I hear you and your point is sound, but there is more to it.
First of all the linked issue, for instance, only appeared under certain network conditions. To write a test reproducing this issue would be pretty tough and might be just as flaky.
The other issue is that we are changing a lot right now, in terms of UI just as much as under the hood, which would require us to constantly rewrite a large part of the tests. This will make much more sense in two or three months from now when the next big wave of refinement is past us.
We now get real user feedback on a sufficiently large scale for the first time, so after we have come close to a point where we are happy with the UI, we shall definitely look into proper testing. Right now we are just changing way too much.
The status quo is not as bad as it might sound, though, as we have a quite solid QA process in place. If this issue is still open by the end of the year, feel free to shout 😄
I do understand what you are saying and I appreciate the time you took to answer my remark in such detail. Still these are all excuses imho. Nothing is more important than having tests.
I agree, yet unfortunately I have to say that there are no resources available it this point. As it's open source, you can of course always contribute some tests :)