experimenter icon indicating copy to clipboard operation
experimenter copied to clipboard

Add sizing warning for single feature experiments

Open eliserichards opened this issue 1 year ago • 0 comments

Thread in #ask-experimenter for context

When one or more multifeature experiments are Live and a user goes to launch a new single feature experiment, they will need to adjust their population sizing to account for the users that are disqualified from being in the multifeature experiments.

We should add an explanation to experimenter.info (https://github.com/mozilla/experimenter-docs/issues/521) and then create a warning (similar to this):

  • For users launching a single feature experiment where the targeted feature already has multifeature experiment(s) that are Live

  • Use the following formula to adjust the sizing:

    if data scientist run normal sizing for feature in “url bar” and get x% as desired typically, but if y% meeting the targeting criterion has been taken by active multiple-feature experiment where ‘url bar’ is one of the feature, they should then use x%/(1-y%) instead?

┆Issue is synchronized with this Jira Task

eliserichards avatar Oct 26 '23 22:10 eliserichards