woocommerce-android icon indicating copy to clipboard operation
woocommerce-android copied to clipboard

[ExPlat] Add A/A test to the app

Open rachelmcr opened this issue 3 years ago • 2 comments

We need to add the following A/A test (experiment) to the app: woocommerceandroid_explat_aa_test_202208

The goal of this experiment is to see if there is any bias in our assignment implementation of ExPlat. Because this is an A/A experiment, there will be no differences between the control and treatment variations, so there's nothing more to be added aside from the test itself (to assign users to a control or treatment group).

You can find details about adding experiments in the ExPlat integration here: https://github.com/woocommerce/woocommerce-android/pull/6055

Experiment design (internal ref): pbxNRc-1QS-p2

rachelmcr avatar Aug 03 '22 10:08 rachelmcr

Fails
:no_entry_sign: Please add a feature label to this issue. e.g. 'feature: stats'

Generated by :no_entry_sign: dangerJS

peril-woocommerce[bot] avatar Aug 03 '22 10:08 peril-woocommerce[bot]

I've just put this task on hold:

Let's wait for the Abacus setup to be done before adding this test. That way, when we add the test to the app we can test that the control/treatment assignment is working as expected before merging it. There's more about doing that testing here: PCYsg-Fq7-p2#4-testing-the-experiment-code

We need this ExPlat/Abacus setup to be done first:

  1. Support for making test assignments for the woocommerceandroid platform
  2. Our woocommerceandroid metrics added to Abacus
  3. The test to be fully set up in Abacus (I'll do this step once the first two are done)

I'll update this issue once that's all done.

rachelmcr avatar Aug 03 '22 12:08 rachelmcr