site-kit-wp icon indicating copy to clipboard operation
site-kit-wp copied to clipboard
verified publisher icon google

Add GA event tracking for the Audience Selection Panel

Open techanvil opened this issue 1 year ago • 1 comments

Feature Description

GA events should be added to keep track of interaction with the Audience Selection Panel.

Do not alter or remove anything below. The following sections will be managed by moderators only.

Acceptance criteria

The following GA events should be tracked for the given actions. See the GA events sheet.

Event Name Category Label Trigger Condition Description/details
change_groups {viewContext}_audiences-sidebar Click User clicks the "Change groups" CTA to open the Selection Panel
audiences_sidebar_view {viewContext}_audiences-sidebar Element visibility User views the Audiences Selection Panel "Sidebar" events named for consistency with the corresponding event for KMW
audiences_sidebar_save {viewContext}_audiences-sidebar count of audiences by category in the format "user:123,site-kit:456,default:789" Click User clicks "Save selection" in the Selection Panel
audiences_sidebar_cancel {viewContext}_audiences-sidebar Click User clicks "Cancel" in the Selection Panel
insufficient_permissions_error {viewContext}_audiences-sidebar Element visibility User views the insufficient permissions error when loading the Selection Panel data
insufficient_permissions_error_request_access {viewContext}_audiences-sidebar Click User clicks "Request access" on the error
data_loading_error {viewContext}_audiences-sidebar Element visibility User views the generic error when loading the Selection Panel data
data_loading_error_retry {viewContext}_audiences-sidebar Click User clicks "Retry" on the error
view_notice {viewContext}_audiences-sidebar-create-audiences Element visibility User views the Audience Creation Notice in the Selection Panel
create_audience {viewContext}_audiences-sidebar-create-audiences one of "new-visitors", "returning-visitors" Click User clicks "Create" for one of the audiences in the notice
view_oauth_notice {viewContext}_audiences-sidebar-create-audiences Element visibility User views the OAuth notice in the Audiences Creation Notice
dismiss_oauth_notice {viewContext}_audiences-sidebar-create-audiences User clicks "Got it" on the OAuth notice
view_notification {viewContext}_audiences-sidebar-create-audiences-success User views the success notification in the Audience Creation Notice
dismiss_notification {viewContext}_audiences-sidebar-create-audiences-success User clicks "Got it" on the notification
auth_error {viewContext}_audiences-sidebar-create-audiences Element visibility User views the OAuth error notice in the Audience Creation Notice when an error occurs during audience creation
insufficient_permissions_error {viewContext}_audiences-sidebar-create-audiences Element visibility User views the insufficient permissions error notice in the Audience Creation Notice when an error occurs during audience creation
insufficient_permissions_error_request_access {viewContext}_audiences-sidebar-create-audiences Click User clicks "Request access" on the error notice
setup_error {viewContext}_audiences-sidebar-create-audiences Element visibility User views the generic error notice in the Audience Creation Notice when an error occurs during audience creation

Implementation Brief

  • [ ] In the ChangeGroupsLink component:
    • Call trackEvent( '${viewContext}_audiences-sidebar', 'change_groups' ) in the openAudiencesSelectionPanel() callback.
  • [ ] In the AudienceSelectionPanel component:
    • Call trackEvent( '${viewContext}_audiences-sidebar', 'audiences_sidebar_view' ) in the onSideSheetOpen() callback.
  • [ ] In the AudienceSelectionPanel/Footer component:
    • Retrieve availableAudiences via the getAvailableAudiences() selector.
    • Add an onSaveSuccess callback prop to the rendered SelectionPanelFooter:
      • Determine the label for the GA event in the format user:123,site-kit:456,default:789 by mapping configuredAudiences to the number of audiences of the given type in the saved selection. The type can be determined by looking up the audienceType for an audience via availableAudiences; 123/456/789 are of course the number of audiences of the given type.
      • Call trackEvent( '${viewContext}_audiences-sidebar', 'audiences_sidebar_save', label ).
    • Add an onCancel callback prop to the rendered SelectionPanelFooter. In the callback, call trackEvent( '${viewContext}_audiences-sidebar', 'audiences_sidebar_cancel' ).
  • [ ] In the AudienceSelectionPanel/ErrorNotice component:
    • Retrieve the selection panel's open state via the CORE_UI AUDIENCE_SELECTION_PANEL_OPENED_KEY value.
    • Create a variable, hasErrors which contains the value of errors.length > 0. This is to use in the useEffect() hook below to avoid passing errors as a dependency as errors will change more often.
    • Add a useEffect() hook.
      • Return early if the panel is closed or hasErrors is false.
      • If hasInsufficientPermissionsError is true, call trackEvent( '${viewContext}_audiences-sidebar', 'insufficient_permissions_error' ).
      • Otherwise, call trackEvent( '${viewContext}_audiences-sidebar', 'data_loading_error' ).
  • [ ] In the AudienceSelectionPanel/RequestAccessButton component:
    • Add an onClick callback prop to the rendered Button. In the callback, call trackEvent( '${viewContext}_audiences-sidebar', 'insufficient_permissions_error_request_access' ).
  • [ ] In the AudienceSelectionPanel/RetryButton component:
    • Add an onClick callback prop to the rendered Button. In the callback, call trackEvent( '${viewContext}_audiences-sidebar', 'data_loading_error_retry' ).
  • [ ] In the AudienceCreationNotice component:
    • Retrieve the selection panel's open state via the CORE_UI AUDIENCE_SELECTION_PANEL_OPENED_KEY value.
    • Add a useEffect() hook.
      • If the panel is open and shouldShowNotice is true, call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'view_notice' ).
    • Amend the onClick handler for the SpinnerButton component:
      • Determine the label for the event, which should be new-visitors or returning-visitors depending on which audience is being created.
      • Call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'create_audience', label ) prior to the call to handleCreateAudience. Chain the two calls using .finally().
    • Add another useEffect() hook.
      • Return early if the panel is closed.
      • If the conditions for showing the SubtleNotification are true (i.e. if ! hasAnalytics4EditScope && ! isEditScopeNoticeDismissed) call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'view_oauth_notice' ).
    • Amend the handleDismissEditScopeNotice() callback:
      • Call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'dismiss_oauth_notice' ) prior to the call to dismissItem. Chain the two calls using .finally().
  • [ ] In the AudienceCreationSuccessNotice component:
    • Retrieve the selection panel's open state via the CORE_UI AUDIENCE_SELECTION_PANEL_OPENED_KEY value.
    • Add a useEffect() hook.
      • If the panel is open and showSuccessNotice is true, call trackEvent( '${viewContext}_audiences-sidebar-create-audiences-success', 'view_notification' ).
    • Amend the onClick handler for the Button component:
      • Call trackEvent( '${viewContext}_audiences-sidebar-create-audiences-success', 'dismiss_notification' ) prior to the call to setValue. Chain the two calls using .finally().
  • [ ] In the AudienceCreationErrorNotice component:
    • Create a variable, hasErrors which contains the value of errors.length > 0. This is to use in the useEffect() hook below to avoid passing errors as a dependency as errors will change more often.
    • Add a useEffect() hook.
      • Return early if hasErrors is false.
      • If hasOAuthError is true, call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'auth_error' ).
      • Otherwise, if hasInsufficientPermissionsError is true, call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'insufficient_permissions_error' ).
      • Otherwise, call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'setup_error' ).
    • Add an onClick callback prop to the rendered Button. In the callback, call trackEvent( '${viewContext}_audiences-sidebar-create-audiences', 'insufficient_permissions_error_request_access' ).

Test Coverage

  • Add JS test coverage for the above changes where tests exist for the affected components.

QA Brief

Changelog entry

techanvil avatar Oct 11 '24 15:10 techanvil

IB ✅

nfmohit avatar Oct 18 '24 18:10 nfmohit

QA Update ⚠

Tested this and while the events are being tracked accordingly, I do have a couple of queries:

ITEM 1: For the audiences_sidebar_save event, we are expecting the label to be a count of audiences by category in the format "user:123,site-kit:456,default:789"

My question is what do the numbers represent? What is the expectation for these? Currently during my testing, I had "user:0,site-kit:2,default:0" I am unable to verify the accuracy of those numbers.

Image

ITEM 2: For the dismiss_oauth_notice event, it would be when User clicks "Got it" on the OAuth notice. Per my testing, I could not see the 'Got it' button from the oauth_notice. Could you double check the attached image and let me know if that's the correct notice to check or we indeed do not have a 'Got it' button?

Image

ITEM 3: For the view_notification event, it triggers when user views the success notification in the Audience Creation Notice.

Let's say there are 2 audiences to be created: New and Returning

  • If I create for the 'New' audience, the event is fired accordingly.
  • Now, if I go on to create for 'Returning' audience, the success message remains but there is a no new firing of the event. Should we be firing it one more time?

https://github.com/user-attachments/assets/1a0fe227-14a1-4b70-a39f-37a57aa298a1

kelvinballoo avatar Nov 06 '24 09:11 kelvinballoo

Thank you for sharing your observations, @kelvinballoo !

ITEM 1: For the audiences_sidebar_save event, we are expecting the label to be a count of audiences by category in the format "user:123,site-kit:456,default:789"

My question is what do the numbers represent? What is the expectation for these? Currently during my testing, I had "user:0,site-kit:2,default:0" I am unable to verify the accuracy of those numbers.

The numbers represent the number of audiences under each category. In your case, you seem to have saved the audience selection with 0 user audiences, 2 Site Kit audiences, and 0 default audiences. Does that help?

ITEM 2: For the dismiss_oauth_notice event, it would be when User clicks "Got it" on the OAuth notice. Per my testing, I could not see the 'Got it' button from the oauth_notice. Could you double check the attached image and let me know if that's the correct notice to check or we indeed do not have a 'Got it' button?

The oauth_notice actually refers to this notice: Image

The notice you're referring to in your screenshot will have the following tracking criteria:

auth_error {viewContext}_audiences-sidebar-create-audiences Element visibility User views the [OAuth error notice](https://www.figma.com/design/7pSrkEy8t00BcYRAi9LjjH/Audience-Segmentation?node-id=1821-26925&node-type=frame&t=PNTGwXm0OYTItlYB-0) in the Audience Creation Notice when an error occurs during audience creation

ITEM 3: For the view_notification event, it triggers when user views the success notification in the Audience Creation Notice.

Let's say there are 2 audiences to be created: New and Returning

  • If I create for the 'New' audience, the event is fired accordingly.
  • Now, if I go on to create for 'Returning' audience, the success message remains but there is a no new firing of the event. Should we be firing it one more time?

With the way it is currently built, the notice doesn't reappear once it is already visible, hence, the event is also tracked once. It might be worth checking with @techanvil to see if the notice should reappear, and if so, a new issue is warranted here.

Please let me know if you have any other questions, thanks!

nfmohit avatar Nov 07 '24 09:11 nfmohit

ITEM 3: For the view_notification event, it triggers when user views the success notification in the Audience Creation Notice. Let's say there are 2 audiences to be created: New and Returning

  • If I create for the 'New' audience, the event is fired accordingly.
  • Now, if I go on to create for 'Returning' audience, the success message remains but there is a no new firing of the event. Should we be firing it one more time?

With the way it is currently built, the notice doesn't reappear once it is already visible, hence, the event is also tracked once. It might be worth checking with @techanvil to see if the notice should reappear, and if so, a new issue is warranted here.

Please let me know if you have any other questions, thanks!

Thanks @kelvinballoo and @nfmohit. As Nahid has pointed out the notice is already visible, there's no need to hide/reshow it in response to the second audience creation and we don't need to track an additional event in the above scenario.

When it comes to analysing the GA events we should be able to recognise that a second audience was successfully created within the span of the current selection panel opening as we have events for the panel opening, the Create CTA being clicked, and the success notification being shown/dismissed and the error notification being shown, all of which should be sufficient to reconstruct the journey.

techanvil avatar Nov 07 '24 10:11 techanvil

QA Update ⚠

Hi @nfmohit ,

ITEM 1: It does help but there seems to be a discrepancy between the audiences that I saved and what is being tracked. If you see in my previous screenshot, the audience selected was only 'New Visitors' but it registered 2. It should have been 1. I have documented other discrepancies in the attached video for you to review.

https://github.com/user-attachments/assets/ee9515ee-e9ca-476f-986c-4accc95729d4

ITEM 2: I was trying to look into steps to re-create that oAuth notice but could not successfully find the steps from other tickets. Could you elaborate on the steps to get the oauth_notice that you pointed out please.

ITEM 3: Noted on that.

kelvinballoo avatar Nov 07 '24 18:11 kelvinballoo

Thank you for the follow-up, @kelvinballoo!

ITEM 1: It does help but there seems to be a discrepancy between the audiences that I saved and what is being tracked. If you see in my previous screenshot, the audience selected was only 'New Visitors' but it registered 2. It should have been 1. I have documented other discrepancies in the attached video for you to review.

I investigated, was able to replicate the issue, and opened a follow-up PR.

ITEM 2: I was trying to look into steps to re-create that oAuth notice but could not successfully find the steps from other tickets. Could you elaborate on the steps to get the oauth_notice that you pointed out please.

You should be able to see this by deleting the wp_googlesitekit_additional_auth_scopes option from the PREFIX_usermeta table for your user.

nfmohit avatar Nov 08 '24 10:11 nfmohit

Back to you for another pass, @kelvinballoo!

techanvil avatar Nov 08 '24 15:11 techanvil

QA Update ✅

Tested ITEM 1 and the audiences saved are now reflecting accordingly on the label.

Image

Image

Image

Image

Image

Other than that, all the events are being tracked accordingly too as detailed further down. The screenshots include ITEM 2 as well.

Moving ticket to approval.

  • change_groups

    Image

  • audiences_sidebar_view

    Image

  • audiences_sidebar_save

    Image

  • audiences_sidebar_cancel

    Image

  • insufficient_permissions_error (audiences-sidebar)

    Image

  • insufficient_permissions_error_request_access (audiences-sidebar)

    Image

  • data_loading_error

    Image

  • data_loading_error_retry

    Image

  • view_notice

    Image

    Image

    Image

  • create_audience

    Image

    Image

  • view_oauth_notice

    Image

  • dismiss_oauth_notice

    Image

  • view_notification

    Image

  • dismiss_notification

    Image

  • auth_error

    Image

  • insufficient_permissions_error (audiences-sidebar-create-audiences)

    Image

  • insufficient_permissions_error_request_access (audiences-sidebar-create-audiences)

    Image

  • setup_error (audiences-sidebar-create-audiences)

    Image

kelvinballoo avatar Nov 11 '24 13:11 kelvinballoo