New A/B Experimentation Setting

[AlexandraKemperMS]

Feature Request

At this point, we have overloaded the "Experimental Features" setting and need a separate setting for A/B experimentation. The experimental features setting was originally created to roll out beta features to users.

With the A/B testing, we need a new setting. This setting should have three options:

• Default: The A/B service determines whether the user is in the treatment or control group for an experiment.
  • Description: You may get included in proposed enhancements and/or features.
• Enabled: They are hard coded into the treatment experience in code. This lets any user get access to the treatment experience, no matter what group they are assigned to by the ExP service.
• Disabled: They are, in code, opting out of any experiments or the potential of being in an experiment.

Since we usually do not have multiple A/B experiments running at once, we do not need multiple settings to opt in/out (like Python has). We can wordsmith what the descriptions would say with the UI/UX team.

[sean-mcmanus]

I'm not understanding why we need a new setting and can't just change the existing setting's setting from "disabled" to "default" (a new 3rd value), i.e. I think we should do that instead. The meaning of the existing experimentalFeatures settings seems valid still and not really in conflict with the A/B experiment.

[AlexandraKemperMS]

When I talked to Bob about this he had mentioned that we still want to keep the original setting since we might have "beta" features that we want to release as an experimental feature. These are separate from A/B experiments. You can see other extensions do this as well:

[sean-mcmanus]

@AlexandraKemperMS @bobbrow I'm still not understanding. We can still have beta non-A/B experimental features with the one setting still.

• Disable: No experimental
• Enable: All experimental
• Default: beta experimental features are disabled, A/B experimental features are subject to the "rollout" randomness I don't see a good reason for a user to want to have different behaviors between the two either.

[github-actions[bot]]

This feature request is being closed due to insufficient upvotes. Please leave a 👍-upvote or 👎-downvote reaction on the issue to help us prioritize it. When enough upvotes are received, this issue will be eligible for our backlog.

[github-actions[bot]]

This feature request is being closed due to insufficient upvotes. Please leave a 👍-upvote or 👎-downvote reaction on the issue to help us prioritize it. When enough upvotes are received, this issue will be eligible for our backlog.