tracker icon indicating copy to clipboard operation
tracker copied to clipboard

Develop A/B testing strategy

Open h701h opened this issue 3 years ago • 1 comments

As a product manager I want to be testing out Tracker functionality in production with real users. As a result we need to pick a A/B testing strategy and some tech to accomplish this.

There are a few paths forward. I want to make sure we are on the same and right page.

Options

  1. Freeze the database
  2. https://github.com/jsdom/jsdom#basic-usage
  3. instio feature flags vs client side feature flags.
  4. kiali version splitting
  5. Flagger (Deployment options)

Start with A/A testing

  • [x] Understand requirements
  • [x] Technologies decision
  • [ ] Dedicated B testers - TBS testers - CCCS testers - Set controls for testers - Set scan guardrails for testing -Potential UAT 'issues capture' doc: https://056gc.sharepoint.com/:w:/r/sites/StrategyDesign-StratgieetConception/Shared%20Documents/Tracker%20-%20Track%20Digital%20Security/Tracker%20-%20UAT%20Issues%20Log.docx?d=w3123844ae2674c4a88ab4e55e268a45d&csf=1&web=1&e=2tNBtR
  • [ ] Implement in GKE (alpha)
  • [ ] Implement in Azure (staging, prod)
  • [ ] Reporting/findings - user interviews?
  • [ ] B-testing toggle on account page (self-serve)
  • [ ] Documentation

h701h avatar Jun 01 '22 14:06 h701h

Definitely take a hard look at Flagger for this.

sleepycat avatar Jun 03 '22 13:06 sleepycat