Research item: anonymous usage analytics

[alexellis]

Expected Behaviour

Track usage of commands (verbs), parameters (used), build times and other anonymous data that can help us understand usage and improve the CLI.

Remit:

• This would only cover the CLI usage - this would not instrument the back-end services.

• Opt-in by default, opt-out of collection via environmental variable in a similar fashion to VSCode and Brew.

.bash_profile

export OPEN_FAAS_TELEMETRY=0

• We must explain how data is used, how long it is kept for etc and make sure it's in keeping with data protection.

• Data must be anonymised

Current Behaviour

Relying on unsolicited feedback / word of mouth

Possible Solution

Google Analytics (subject to costing out) - has a Golang library. https://github.com/google/google-api-go-client/blob/master/analytics/v3/analytics-gen.go

Initial work would only be a PoC

[johnmccabe]

I'll pick this up, have done something similar in the past.

[johnmccabe]

Derek assign: johnmccabe

[johnmccabe]

Have this working locally, will push up a PR asap - heres a screengrab of a Data Studio report, generated with a dev build - thanks to @kinghuang and @burtonr for being my guinea pigs today.

[johnmccabe]

@alexellis can you have a think about what else might be useful to capture

Currently have

• Country
• Random UUID for telling an existing user from a new one
• OS
• CLI version
• CLI verb used
• Language when the subcommand uses it, unset otherwise

Other ideas

• Number of deployed functions
• Total invocations across all functions
• Mean invocations across all functions
• Whether the gateway is running on localhost vs. a remote host (thinking this should just be a bool, as I don't think we want to start doing lookups on IPs).
• Backend in use, Swarm, Kubernetes, Hyper etc (this one is a no-brainer I think)

[johnmccabe]

Also, do you want this opt-in, or opt-out.

Adding a flag and support for setting an envvar that sets your preference wrt letting the project capture the basic usage info above.

[johnmccabe]

Derek add label: hacktoberfest

[ghost]

@alexellis @johnmccabe This looks great. Should we also consider:

• Number of replicas
• Duration of invocations
• API connected? Yes/No

In reference to your other ideas section

• Number of deployed functions != sample functions

[alexellis]

@briansiemens - John's work right now is not around the daemon/server but at the faas-cli level.

I'd say opt out via ENV_VAR to follow the model of Brew and VSCode.

We can report functions deployed / invoked but only really if someone uses faas-cli list

In order to track back-end used, we should return a header via the gateway for system functions - or introduce an info endpoint. People should also be able to turn off the clue being leaked via the gateway for when they don't want to reveal this information. (separate proposal)

[johnmccabe]

@briansiemens can you clarify what you're mean by API connected.

[vegasbrianc]

@johnmccabe Sorry was logged into the wrong account. I was just brainstorming if we could see if something like Kong or some other service is connected or not.

[alexellis]

Additional useful data:

• Count of URLs in login section and whether they use HTTP/s
• Are they localhost or remote (don't pick up the actual URL)
• Is --parallel being used?

Let's checkout the new commands we've added too.

I need to checkout any legal implications or data protection notices we need to include before merging. Conceptually looks great.

On first run - should we write a $HOME/.openfaas/privacy_notice file?

[alexellis]

Bump on this. If we can do this in a GDPR-friendly fashion then I think we should and it will help greatly with understanding usage of the flags/templates etc.