Should tests succeed if connection times out?

[LukeOwlclaw]

I just tried CreateFolderTest. It succeeded. I noticed that several connection times outs occurred before I noticed that my connection was blocked.

I looks like timeouts are "accept" on purpose:

https://github.com/owncloud/android-library/blob/develop/test_client/tests/src/com/owncloud/android/lib/test_project/test/RemoteTest.java#L54

https://github.com/owncloud/android-library/blob/develop/test_client/tests/src/com/owncloud/android/lib/test_project/test/CreateFolderTest.java#L71

I do not understand the reasoning here. IMO, the tests require a working connection to the server, otherwise functionality cannot be tested. So if the connection breaks, the test should fail.

[davivel]

That was done that way to avoid blocking merges due to problems in server side, since the tests should test the client library. But yes, it was cheating.

Even better would be that the tests did not use a real server, but some kind of network mock-up. But that's so out of our scope right now...

Feel free to update the tests as you suggest.

[LukeOwlclaw]

Are tests performed automatically before merging? And is merging prevented if tests fail? That sounds like a good idea, but also quite complex!

[davivel]

Yes, they are done before merging. The prevention is manual - Github reports in the bottom of the PRs if Travis tests was passed. In the case of the library, the Travis tests iinclude the tests in test_project/test

[JuancaG05]

Outdated, closing here 🚀