application-services icon indicating copy to clipboard operation
application-services copied to clipboard

[Data Review] for Record metrics for "my device has an expired endpoint"

Open data-sync-user opened this issue 4 years ago • 0 comments

For each of Android, iOS and Desktop we should record the state of push for this device. Specifically, once per session, after the list of FxA devices has been fetched, we should record which of the following states our device is in:

  • My device apparently doesn't exist! (This should be rare, but would be bad if we discovered high numbers)
  • My push end-point is good
  • My push end-point is expired.

We should talk to someone (Travis?) to work out:

  • How to record this on each platform (I think the answer will be "glean" for mobile, not clear for desktop. Glean sounds ideal so we can reuse the same dashboard, but not clear how much work that would be for desktop?)
  • Ensure that once data comes in, we will be able to notice the transitions between these states? - note that the absolute values are only partially interesting, what's more interesting is how often we transition from good to bad (and vice-versa) and how long it was since the last transition, etc.

Questions we will want to answer are things like:

For each OS:

  • What %age of active devices are in a bad state at any one time.
  • On average, how long is a device in a "good" state before it goes "bad".
  • On average, how many startups does it take for a device to become good? ("how long does it take" doesn't seem like a good metric - if a device is used once a month but repairs itself on the very next restart, it's not useful to say it was in a bad state for a month)
  • How many devices never repair?
  • How many devices don't actually report this (ie, are so fundamentally broken WRT FxA/Push that we know they are using FxA, but they never record these metrics)

This will end up being 3 tickets (one for each platform), but we should have come up with a strategy we can share before starting work.

┆Issue is synchronized with this Jira Task ┆epic: Fix Push ┆sprintEndDate: 2021-11-24

data-sync-user avatar Oct 08 '21 00:10 data-sync-user