Allow manually assign users to A/B test groups from client app or Admin SDK

[zmeid]

What feature would you like to see?

Allow manually assign users to A/B test groups from client app or Admin SDK

How would you use it?

The purpose of that request is basically to handle the similar edge cases explained here:

https://towardsdatascience.com/4-mistakes-that-i-made-while-running-a-b-tests-with-firebase-remote-config-f7b17f18b34a -> 4. Managing re-installs and multiple devices

Steps how we hit edge cases:

1. We have 2 user groups to test a major new feature(Baseline: GroupA - New feature users: GroupB)
2. Let's say a user is assigned to GroupB and has the new feature.
3. We set a flag on our servers to inform the server that the user was selected to be part of GroupB by Firebase. So, the server returns the new data response for that user's API calls that is required by the new feature that we are testing.
4. User reinstalls the app and falls into GroupA OR user logs on another device where the app instance was assigned to GroupA.
5. On our servers we still have the flag that indicates we served the new feature to the user so we will continue to serve it.
6. But, Firebase A/B will think that this user is not part of GroupB.
7. The problem: This will affect the results of the experiment. Because the user will be populating Firebase A/B testing with wrong group assignment data.

In A/B testing, user grouping is tied to app instance IDs, not user IDs. We want to be able to tell Firebase A/B testing that the user was previously assigned to GroupB and should continue to be part of GroupB.

Workaround for re-installs:

• Set an activation flag for the A/B test on Firebase A/B testing console. For ex: "ok_for_experiment".
• Fire the event "ok_for_experiment" only if we determine that this user was not part of the GroupB experiment previously.
• So, Firebase won't assign that user to groups randomly and the user will be out of the experiment.
• Cons: We will not have further analytics data for the experiment from this user since we won't let the user re-join to experiment
• Pros: User will not populate the experiment with false data(when A/B testing thinks that user is not part of the new feature where actually is)

Workaround for multiple log ins: When user logs on another device and if that device is assigned to a different group other than user's group:

• Remove the Firebase Instance ID from that device so Firebase removes the data associated with that Firebase instance ID(Hoping it works this way).
• Do not fire the event "ok_for_experiment" on that device so Firebase won't re-assign that app instance to any groups.

Not really sure if that workaround for multiple log-ins would work. Because I see that note:

https://firebase.google.com/docs/projects/manage-installations#delete-installation Google Analytics for Firebase uses its own form of ID to keep track of analytics data. Deleting a Firebase installation ID does not delete Analytics data. Learn how to delete Analytics data associated with end users.

Since deleting instance ID does not delete Analytics data, will it remove the app instance's association with the Firebase A/B testing(because A/B testing pulls the data from analytics)?

[google-oss-bot]

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.