fenix icon indicating copy to clipboard operation
fenix copied to clipboard

For #26191 - Add telemetry for jump back in onboarding message

Open Alexandru2909 opened this issue 3 years ago • 1 comments

Added telemetry for jump back in cfr show, dismiss and cancel.

Pull Request checklist

  • [ ] Tests: This PR includes thorough tests or an explanation of why it does not
  • [ ] Screenshots: This PR includes screenshots or GIFs of the changes made or an explanation of why it does not
  • [ ] Accessibility: The code in this PR follows accessibility best practices or does not include any user facing features. In addition, it includes a screenshot of a successful accessibility scan to ensure no new defects are added to the product.

QA

  • [x] QA Needed

To download an APK when reviewing a PR (after all CI tasks finished running):

  1. Click on Checks at the top of the PR page.
  2. Click on the firefoxci-taskcluster group on the left to expand all tasks.
  3. Click on the build-debug task.
  4. Click on View task in Taskcluster in the new DETAILS section.
  5. The APK links should be on the right side of the screen, named for each CPU architecture.

GitHub Automation

Fixes #26191

Alexandru2909 avatar Jul 27 '22 07:07 Alexandru2909

Waiting for https://github.com/mozilla-mobile/fenix/pull/26203 to land.

Alexandru2909 avatar Aug 09 '22 11:08 Alexandru2909

This pull request has conflicts when rebasing. Could you fix it @Alexandru2909? 🙏

mergify[bot] avatar Aug 18 '22 21:08 mergify[bot]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

  1. What questions will you answer with this data?
  • How many users saw the jump back in onboarding message?
  • After the onboarding message was displayed, how many users dismissed the CFR using the `X` button?
  • After the onboarding message was displayed, how many users cancelled the CFR by touching elsewhere on the screen?
  1. Why does Mozilla need to answer these questions? Are there benefits for users? Do we need this information to address product or business requirements?
  • We need to count the impressions, clicks on the `X` button and clicks elsewhere on the screen to understand how many users see the jump back in cfr and how they interact with it.
  1. What alternative methods did you consider to answer these questions? Why were they not sufficient?
  • No other alternatives.

  1. Can current instrumentation answer these questions?
  • No.

  1. List all proposed measurements and indicate the category of data collection for each measurement, using the Firefox data collection categories found on the Mozilla wiki.

    Note that the data steward reviewing your request will characterize your data collection based on the highest (and most sensitive) category.
Measurement Description Data Collection Category Tracking Bug #
Jump back in cfr shown: Jump back in CFR has been displayed to the user. Category 2 - interaction data https://github.com/mozilla-mobile/fenix/issues/26190
Jump back in cfr dismissed: Jump back in CFR has been dismissed clicking the X button. Category 2 - interaction data https://github.com/mozilla-mobile/fenix/issues/26190
Jump back in cfr cancelled: Jump back in CFR has been cancelled clicking the outside the CFR. Category 2 - interaction data https://github.com/mozilla-mobile/fenix/issues/26190
  1. Please provide a link to the documentation for this data collection which describes the ultimate data set in a public, complete, and accurate way.
  1. How long will this data be collected? Choose one of the following:
  • Up until version 119, with the option to renew at that point.
  1. What populations will you measure?
  • All channels, all locales, all countries.

  1. If this data collection is default on, what is the opt-out mechanism for users?
  • Default Glean SDK opt-out mechanism.

  1. Please provide a general description of how you will analyze this data.
  • Glean.

  1. Where do you intend to share the results of your analysis?
  • 
Only on Glean and with mobile teams.

  1. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?
  • No third-party tools.

Alexandru2909 avatar Aug 23 '22 09:08 Alexandru2909

This pull request has conflicts when rebasing. Could you fix it @Alexandru2909? 🙏

mergify[bot] avatar Aug 25 '22 18:08 mergify[bot]

Request for data collection review form

All questions are mandatory. You must receive review from a data steward peer on your responses to these questions before shipping new data collection.

1. What questions will you answer with this data?


* How many users saw the jump back in onboarding message?

* After the onboarding message was displayed, how many users dismissed the CFR using the `X` button?

* After the onboarding message was displayed, how many users cancelled the CFR by touching elsewhere on the screen?


2. Why does Mozilla need to answer these questions? Are there benefits 
   for users? Do we need this information to address product or business 
   requirements?


* We need to count the impressions, clicks on the `X` button and clicks elsewhere on the screen to understand how many users see the jump back in cfr and how they interact with it.


3. What alternative methods did you consider to answer these questions? Why were they not sufficient?


* No other alternatives.


4. Can current instrumentation answer these questions?


* No.


5. List all proposed measurements and indicate the category of data 
   collection for each measurement, using the Firefox data collection 
   categories found on the Mozilla wiki.
   
   Note that the data steward reviewing your request will characterize your
    data collection based on the highest (and most sensitive) category.

Measurement Description Data Collection Category Tracking Bug # Jump back in cfr shown: Jump back in CFR has been displayed to the user. Category 2 - interaction data #26190 Jump back in cfr dismissed: Jump back in CFR has been dismissed clicking the X button. Category 2 - interaction data #26190 Jump back in cfr cancelled: Jump back in CFR has been cancelled clicking the outside the CFR. Category 2 - interaction data #26190

6. Please provide a link to the documentation for this data collection 
   which describes the ultimate data set in a public, complete, and 
   accurate way.


* [https://dictionary.telemetry.mozilla.org/apps/fenix
  ](https://dictionary.telemetry.mozilla.org/apps/fenix)


7. How long will this data be collected? Choose one of the following:


* Up until version 119, with the option to renew at that point.


8. What populations will you measure?


* All channels, all locales, all countries.


9. If this data collection is default on, what is the opt-out mechanism for users?


* Default Glean SDK opt-out mechanism.


10. Please provide a general description of how you will analyze this data.


* Glean.


11. Where do you intend to share the results of your analysis?


* Only on Glean and with mobile teams.


12. Is there a third-party tool (i.e. not Telemetry) that you are proposing to use for this data collection?


* No third-party tools.

Data Review

  1. Is there or will there be documentation that describes the schema for the ultimate data set in a public, complete, and accurate way?

Yes, through the metrics.yaml file and the Glean Dictionary

  1. Is there a control mechanism that allows the user to turn the data collection on and off?

Yes, through the "Send Usage Data" preference in the application settings

  1. If the request is for permanent data collection, is there someone who will monitor the data over time?

N/A, collection set to end or be renewed by version 119

  1. Using the category system of data types on the Mozilla wiki, what collection type of data do the requested measurements fall under?

Category 2, Interaction data

  1. Is the data collection request for default-on or default-off?

default-on

  1. Does the instrumentation include the addition of any new identifiers?

No

  1. Is the data collection covered by the existing Firefox privacy notice?

Yes

  1. Does the data collection use a third-party collection tool?

No

Result

data-review+

rocketsroger avatar Sep 01 '22 16:09 rocketsroger

Thanks for picking up this data-review @rocketsroger :)

travis79 avatar Sep 06 '22 13:09 travis79