How usable is topcoat-server for other projects?

[addyosmani]

Hey Andrei,

I was just wondering - how coupled is the Telemetry integration in topcoat-server to Topcoat itself? The server setup you have here for bench.topcoat.io is pretty fantastic and I think there would be great value in us making it more easily accessible to other developers.

Would you be interested in documenting this process or taking us through what might need to be done to generalize this work a little further?

[piatra]

Hi and thanks!

Shortstory is that we have a Jenkins machine (triggered by a new Topcoat release i.e. version bump in the package.json) that runs Telemetry tests on each component that is part of the release.

The bench.topcoat.io graphs are there to easily spot performance issues with any of our components. So I think it could easily be any other tool. We have investigated using jsperf as well, and I have personally tried robohornet (but it doesn't work for mobile from what I know - our Telemetry tests run on a nexus 4 device). Topcoat server started as a test suite over benchmark.js actually.

It is very usable for other projects (at least that is my intention). I made a screencast http://www.youtube.com/watch?v=Mhw4Sf1WWwQ on how you can use topcoat-server and it has nothing to do with Topcoat itself. Agreed there might be some polishing needed.

The topcoat jenkins configurations are also open sourced here https://github.com/topcoat/topcoat-ci-config . Any other project will have to write its own jobs for jenkins but they provide a nice start.

Does that answer your question ?

[addyosmani]

Thanks for the detailed overview of where topcoat-server is, @piatra! I really appreciate it. I've just been looking through your screencast and think it's a great first draft to help others get setup with the telemetry side of testing.

Some feedback in case it's useful at all:

• The screencast could give an overview at the start of the high-level steps needed to get topcoat-server and the jenkins box setup - maybe this might be a separate screencast. Perhaps that makes more sense. I'm sure that something covering setting this up would be useful to others.
• Is there any other backend or database you need to setup to get everything working?

An end-to-end setup guide in the readme would be amazing.

[piatra]

Sounds like a great idea :) I'll give it a try and keep you posted on the updates.

[piatra]

@addyosmani https://github.com/topcoat/topcoat/wiki/Topcoat-performance-testing

Check out the screencast there I hope people will find it useful. Sorry for my lack of enthusiasm it is quite the opposite but I was nervous haha. It should give a good idea about how you can use it.

The wiki page can be used to gather FAQ.

For any questions from users we have a google group https://groups.google.com/forum/#!forum/topcoat or the Topcoat issue list which I also mentioned at the end of the screencast. I'll stress again that you only have to download a chromium tarball and no other configuration or build step are necessary hopefully people won't get scared away :)