Phillip Baker
Phillip Baker
Based on this article on [A/B testing](http://www.evanmiller.org/how-not-to-run-an-ab-test.html) (and other similar articles), for a traditional A/B experiment: - sample sizes should be decided before starting an experiment - significance of the...
As #198 and #199 pointed out, having test support for rspec/minitest/integration tests for common patterns would be very helpful.
Dummy controllers/mailers probably belong in the dummy rails apps. See: - https://github.com/assaf/vanity/blob/master/test/experiment/ab_test.rb#L3 - https://github.com/assaf/vanity/blob/master/test/rails_test.rb#L3 - https://github.com/assaf/vanity/blob/master/test/rails_dashboard_test.rb#L3 - https://github.com/assaf/vanity/blob/master/test/rails_test.rb#L191
Currently both `def test_....` and spec-style syntax is used, we should consolidate for consistency.
The Rails generator currently only creates SQL migrations, we should include a command for generating the controller for the dashboard, tracking image and JS callbacks or move it to a...
Instead of using `set_...`, `get_...` and `is_...`, we should switch to `...=`, a standard getter, and `...?`. For example: https://github.com/assaf/vanity/blob/master/lib/vanity/adapters/abstract_adapter.rb#L70
It'd be really cool to have Redis as a primary adapter with a SQL adapter as a secondary. This would allow fast response times (Redis), while also facilitating long-term data...
Consider that configuration should be [separate from code](http://12factor.net/config), which might include features like: - selectively enabling and disabling tests (without modifying code) - default values for experiments (see apartmentlist/vanity#2) -...
It'd be sweet to add [multivariate testing](http://en.wikipedia.org/wiki/Multivariate_testing#In_internet_marketing). Experiments would have to be "grouped" or somehow notified that they're on the same page/part of a larger experiment to calculate the overall...