gorse
gorse copied to clipboard
Support AB tests of configuration
Is your feature request related to a problem? Please describe. I am trying to tune the configuration and find which one works best for my users. Changing configuration for all users may be risky and could lead to money loss (if the new configuration is not as good).
Describe the solution you'd like I want to be able to start AB testing multiple configurations.
Describe alternatives you've considered I have considered having multiple Gorse apps with different configurations and have an AB test logic in the web server outside of Gorse (variants, user-id resolution, time range).
Additional context I would like to create an AB test in a time range (in days) for X% of users. AB testing should be stateless. Each variant will have a configuration to be used. I want to choose the percentage and number of variants. Choosing 30% and 3 variants means that there will be 5% A variant, 5% A control variant, 5% B variant, and so on. Gorse API has a new endpoint (e.g. /{user-id}/abtest) that returns the user's test variant. I can provide more details about implementation if needed.