Linkage Monitor analytics

[elharo]

Write a one pager on how to collect analytics for linkage monitor:

1. Number of repos installed in versus number of repos available (latter is inventory project?)
2. Number of runs
3. Number of errors caught

Possibly guard so we only collect on google owned repos. Possibly bucket by organization.

[elharo]

see https://g3doc.corp.google.com/company/teams/pwg/analytics/ga/oandoReview.md?cl=head

[elharo]

Next steps:

1. Privacy design doc in Eldar
2. Create a Google Analytics account to include in the review ticket.

This is before we add any code to support this.

[suztomo]

My memo:

How Google Cloud Eclipse handles this

Upon SDK installation, it calls AnalyticsPingManager.getInstance().sendPing(AnalyticsEvents.CLOUD_SDK_INSTALL_SUCCESS);

https://github.com/GoogleCloudPlatform/google-cloud-eclipse/blob/3e0263dec7d5faa6f6e4b4981a8a5247fb48446e/plugins/com.google.cloud.tools.eclipse.sdk/src/com/google/cloud/tools/eclipse/sdk/internal/CloudSdkInstallJob.java#L78

Eclipse provides framework to schedule jobs. In AnalyticsPingManager.java:

  @VisibleForTesting
  final Job eventFlushJob = new Job("Analytics Event Submission") {
    @Override
    protected IStatus run(IProgressMonitor monitor) {
      while (!pingEventQueue.isEmpty() && !monitor.isCanceled()) {
        PingEvent event = pingEventQueue.poll();
        showOptInDialogIfNeeded(event.shell);
        sendPing(event);
      }
      return Status.OK_STATUS;
    }
  };

It sends JSON data to collectionUrl.

        String json = jsonEncode(pingEvent);
        int resultCode = HttpUtil.sendPost(collectionUrl, json, "application/json");

Analytics API

collectionUrl is

https://github.com/GoogleCloudPlatform/google-cloud-eclipse/blob/2f2d4c8d6c83d884ebfe09d0cc1eb55b424d0f93/plugins/com.google.cloud.tools.eclipse.usagetracker/src/com/google/cloud/tools/eclipse/usagetracker/AnalyticsPingManager.java#L56

The URL contains "legacy" in it. Should we use non-legacy API?

What events do we want to cover?

• repository name
• pull request URL
• started
• succeeded
• failed Do we care how check failed?
  • Internal error or genuine dependency conflicts

[elharo]

Our implementation will likely be simpler. We're not collecting user data of any kind and can ask for consent at install time so we don't need to show any dialogs.

[elharo]

I expect we'll update com.google.cloud.tools.dependencies.linkagemonitor.LinkageMonitor so we run it from the script like

java -jar ${JAR} --send-analytics com.google.cloud:libraries-bom

If the flag is present, we ping GA. If it isn't, we don't.

Since no current script contains this flag, explicit action is required to turn it on.

[elharo]

https://github.com/GoogleCloudPlatform/cloud-opensource-java/wiki/Linkage-Monitor-Analytics

[suztomo]

Thank you.

We will need a client key for Google analytics that is not published in the Github repository but is bundled into the jar file as part of the build process.

This is an interesting challenge to tackle. I'm going to create a document on how to implement this. (Let me know if you already have a plan)

[elharo]

Don't bother. We've done this before in our other projects that use GA. No need to reinvent that now.

[suztomo]

Good to hear. Then we'll rely on the existing method.

[elharo]

Relevant existing code for replacing constants when kokoro builds the binary:

https://github.com/GoogleCloudPlatform/google-cloud-eclipse/search?q=firelog.api.key https://source.corp.google.com/search?q=%20FIRELOG_API_KEY%20eclipse%20&sq=package:piper%20file:%2F%2Fdepot%2Fgoogle3%20-file:google3%2Fexperimental