wordpress-mobile
Android Analytics for Auto Opt-In To Gutenberg Are Incorrect
Describe the bug On Android, the analytics showing that users are being automatically opted into Gutenberg appear to be firing way too often. Even though we shouldn't still be auto-opting in very many people, they've stayed relatively steady ever since we started opting people in automatically. iOS's numbers, by contrast declined as expected and are now about 1/200th of the numbers Android is reporting.
To Reproduce
Look at the analytics for the wpandroid_gutenberg_enabled event compared to the wpios_gutenberg_setting_enabled event.
Expected behavior Our analytics should show consistent behavior between Android and iOS.
It feels like this is almost certainly an analytics bug, but there is a chance that something could not be working as we expect with the Android rollout code for Gutenberg.
It looks like around 75% of these events are coming from site creation, and 25% are coming on opening a block post.
Looks like the platforms started diverting (in terms of uniques fired) late in March 2020, although they were aligned for (most of) March 2020.
Removing myself as assignee since I'm not working on this at the moment. @mchowning given this is High Priority, should I or someone else be looking at this, or is it ok to leave unassigned?
Thanks for the ping Paul. I think it's ok to leave unassigned since we have no indications that there is an issue with how Gutenberg opt-in rolled out on Android, but I'll leave it as high priority because I still think we need to look into what is causing this to ensure there's no issue.
I'm going ahead to remove the [Pri] High label. The analytics are still incorrect and it'd be good to fix. However, the fact that this has been an issue for over two years with no urgent requests to fix, makes me lean towards the priority not being high. In addition, the issue doesn't impact users directly. Let me know if you disagree with the prioritisation change!
