Give ourselves better Self-hosted Stats

[hubertdeng123]

Right now, we have a pretty broken system used to gain visibility into our self-hosted install userbase. It’s been difficult to track usage across different versions of our product, and it’s also not clear which features are being used by the majority of our install base.

Primary Goal

Better and more helpful statistics when it comes to gaining visibility of all our self-hosted installs. This will help understand how most people are using self-hosted, and if there are any areas that we should specifically invest in to make sure our users are happy.

What kind of stats are useful?

1. CPU/Memory of the boxes self hosted Sentry is running on
2. Features that self hosted instances are using
3. Geolocation of instances
4. Upgrade metrics, we should potentially every time a user has upgraded, their install id should be the same

How will these stats be surfaced? We have a pipeline in eng-pipes already to query data from BigQuery. We can expand that to include these metrics that come through the beacon. We can then create an api endpoint in eng-pipes/getsentry from where static sites can read off of, to construct a page on the open.sentry.io site.

Tasks

• [x] Add New types of stats
  • [x] Add count of type of Sentry events in last 24h (Profiles, Errors, Transactions, Replays, etc)
  • [x] Add CPU/Memory of self hosted instnaces
  • [x] Add Geolocation of instances
  • [x] ~Add Upgrade metrics~ - decided not valuable enough
• [x] ~Stats endpoint in getsentry/eng-pipes~ - deprioritized in favor of DDM
• [x] ~Frontend page on open.sentry.io/self-hosted/stats~ - deprioritized in favor of DDM

[chadwhitacre]

Should we split out the broadcast work from the beacon work? They're related but separate, no?

[chadwhitacre]

Also, what do you think of moving this to the self-hosted repo to get more visibility with the community over there? We need input from the community to make sure these changes are done in a helpful way and benefit the community, like, we also need a ticket to talk about a public dashboard.

[chadwhitacre]

I feel like this should be a whole epic🔒, beacon is one part of it but so is downloads and broadcast and public dashboard. Need to get specific and flesh this all out.

[hubertdeng123]

I feel like this should be a whole epic, beacon is one part of it but so is downloads and broadcast and public dashboard. Need to get specific and flesh this all out.

Agreed.

[chadwhitacre]

From the call today ... I think this is a great opportunity to dogfood DDM. @hubertdeng123 Can you update the description/subissues to bring this in? Then we should reach out in #discuss-ddm on Slack, so they know we're planning to go this route. It seems quite likely we'll find product improvements to suggest, let's ticket those in GitHub as we hit them, and coordinate with the DDM team to find workaround as needed. This is gonna be great! :D

[chadwhitacre]

Quick notes from call:

1. port beacon to DDM
   1. dual-write to DDM
      1. from user instance for new installs
      2. reflected from SaaS for old installs

[hubertdeng123]

DDM has a retention of 90 days. This will change in the future, but with no timeline. This means we will need to keep the bulk of our data in looker.

[chadwhitacre]

Change to what? Indefinite?

[hubertdeng123]

Yep, our eventual goal is to not have a retention period on metrics

[chadwhitacre]

Decided to scrap plans for realtime external dashboard in favor of DDM, but DDM is not mature enough yet to fully replace Looker so dual-writing for now.

[hubertdeng123]

Going to close this as good for now. New stats are in the dashboard🔒