jsxc
jsxc copied to clipboard
jsxc
Testing
Hey @Zauberstuhl @johangm90 @shaggybb @dimaursu,
one thing bugs me from the total beginning of this project: Testing.
Every time we change something we have to click through the complete application to ensure everything works as expected. This is not only inconvenient, but also error-prone.
Why did I not write some tests in the beginning? Really simple. I was afraid of writing some xmpp mocks. (Now I am more afraid to click through the complete application ;-))
Now I would propose the following:
- start writing some tests after 2.1 was released
- unit tests (is it still a unit test, if you test ui and attached handler together?) with qunit and sinonjs
- run unit tests with phantomjs
- integration tests with two "virtual" users and real xmpp server
I have only experience with qunit in smaller projects, so I have also a couple of questions:
- How to run cross-browser tests? Maybe https://saucelabs.com/opensauce/ ?
- How to create virtual users?
I found the following projects, which maybe fits our needs or could be used as alternative to the above:
- http://nightwatchjs.org
- http://dalekjs.com
- https://github.com/opitzconsulting/uitest.js
- http://casperjs.org
What do you think, or which experience do you have with javascript testing? I appreciate every comment.
Very good point!
I have probably lesser experience then you. I started something with the diaspora fork and qunit (https://github.com/diaspora/jsxc/tree/develop/test). But that hasn't evolved much, yet.
In the next couple of weeks my focus is on getting back to the chat stuff in diaspora and I would love to support you with writing some tests!
Sadly, I cannot answer a single question .. but I think we need this to erase a huge amount of pain!
The easy way out of it is to have a real XMPP server, and 2 real users - that way you can avoid all sort of minor issues with mocking or stubbing out XMPP accounts.
I suppose something like this might be useful - https://github.com/node-xmpp/node-xmpp-server/blob/master/examples/server-and-client.js If an XMPP server is started locally, the tests will be way faster.
@Zauberstuhl Thanks for the link. That is a great summary.
@dimaursu I am not sure if I got you right, but what you are proposing are the manual tests which I did for the past 2 years. I am looking for some kind of automated tests, because I would like to do cross browser and cross server tests.
No, I meant using using some browser automation tool with 2 real users - something like Selenium. What I was trying to say is to avoid mocking and stubbing XMPP servers in Jasmine, or other such JS testing frameworks. By "real users" I meant actual accounts instead of some JS objects.
Testing http://localhost:3000/test/jsxc/jsxc_spec.html Browser does not appear to be WebRTC-capable
FF.
>> global failure
>> Message: ReferenceError: Can't find variable: RTCPeerConnection
Just ran into this. Haven't debugged much yet but we should use something which can speak rtc protocol
A few points I have...
is it still a unit test, if you test ui and attached handler together?that's an integration test for sure. I would try to separate the handler out and test it as a unit with mocha (my favorite) or whatever else you want. Then, run an integration test separately on the same feature from an UI perspective.- Phantomjs doesn't support edge features, which is great if you want backwards compatibility but I think this project cannot support those kinds of browsers. Also please don't use Phantomjs for unit tests, there are many other great solutions such as mocha/qunit as you mentioned etc. I personally use casperjs for integration tests and find it pretty complete. It also has compatibility with SlimerJS which I have actually not tried, but it does support edge features like webrtc because it runs the latest firefox.
- I feel the cross-browser testing might not be worth the effort. Standards are pretty good today and I find that any odd styling issue that occurs are pretty edge-case/rare. I know it sounds bad, but for our sanity, I suggest we just use one headless browser solution and leave the rest to manual testing/github issues. Of course, I could be entirely wrong but if anyone has bright ideas then awesome!
- For virtual users, I would leave that up to the developer/travis configuration to set up their local xmpp server correctly for testing and then add it as an instruction in the docs.
- For testing 2-way communication, I think we should relax and just test sending, and simulate receiving in unit tests. After all, we are not testing the xmpp server, just jsxc.
Sorry I just dived right in here but this is a great project and I'd like to help out.
@krazyjakee Thank you for your thoughts. Sounds like you have some experience with js testing and this is really what we need. I would be grateful if I could ref you if there is some progress on this topic.