experimenter icon indicating copy to clipboard operation
experimenter copied to clipboard

prefFlip experiments should not be able to be sent to Preview as the client rejects natual enrollment

Open data-sync-user opened this issue 8 months ago • 0 comments

[Prerequisites]:

  • Have access to the Nimbus Console.
  • Have a desktop experiment created using the prefFlips feature configuration.
  • Set up the latest Nightly to point to the Stage server and the nimbus-preview collection.
    • Set the messaging-system.log pref to all.

[Steps to reproduce]:

  1. Navigate to the experiment page from prerequisites.
  2. Click the Launch to Preview button.
  3. Wait for the Experiment to be displayed[ |https://firefox.settings.services.allizom.org/v1/buckets/main/collections/nimbus-preview/records]+here+.
  4. Open the Firefox browser from prerequisites.
  5. Open the Browser Console and search for the experiment slug.

[Expected result]:

  • Step 2: An error message is displayed stating that prefFlip experiments can’t be sent to preview.

[Actual result]:

  • Step 2: The experiment is successfully sent to Preview.

[Notes]:

  • Attached is a screen recording of the issue.
  • Once the experiment is sent to Preview the client enroll in it naturally because it throws an error. !image-20240612-134950.png|width=892,height=76,alt="image-20240612-134950.png"!
    • While forced enrollment is still possible, bucketing as well as targeting are not testable with this method.
  • I’m marking this as an S2 because it blocks a main QA flow and people using the system would think that Preview works just like in other Experiments or Rollouts.

┆Issue is synchronized with this Jira Bug ┆Attachments: image-20240612-134950.png

data-sync-user avatar Jun 12 '24 13:06 data-sync-user