autotune
autotune copied to clipboard
Include a test for A/B testing usecase as part of PR check
Include a test for A/B testing usecase.
In this usecase, the user provides a deployment name, two docker images and metrics to be monitored during the course of the experiment. The goal is to deploy the docker images in two separate trials respectively, one after the other. It will monitor the given metrics during both the trials and report the results at the end of each trial. The results from each of the trial can then be used to compare the two trials and determine if there has been a variation in the performance objective and the quantum of the same.