Export data to BigQuery

[Angelodaniel]

Problem Statement

As customers often extract their data into their own datastorage it's important that we can integrate into their existing way of working in this case with BigQuery.

Also see this: https://github.com/getsentry/sentry/discussions/52297#discussioncomment-6379561

One thing that we've tried to build a few times with the Sentry APIs is trying to effectively export data into something like BigQuery or any other data warehouse since there are usually a lot of ad-hoc queries we want to run on say a given release, or how a release compares to another release. With the existing APIs this is extremely challenging, often times you can get the raw event, but trying to back fill the information that gets created for the issue into those events ends up making this kind of a challenge just not worth it. I would push to consider how users of Sentry could export their data in a way that would give them the ability to run rich queries in external tools, this could provide for an extremely large set of use cases.

Solution Brainstorm

Customer is looking for a Sentry equivalent to https://firebase.google.com/docs/crashlytics/bigquery-export.

Product Area

Settings - Integrations

┆Issue is synchronized with this Jira Improvement by Unito

[getsantry[bot]]

Routing to @getsentry/product-owners-settings-integrations for triage ⏲️

[getsantry[bot]]

Assigning to @getsentry/support for routing ⏲️

[peterHakio]

We could very much use this. What are the chances it could prioritised?

[sentaur-athena]

This is not in our plans in near future.

[VeVarunSharma]

This is not in our plans in near future.

@sentaur-athena is there any way to export sentry data out, or connect it to something like Posthog? :)

[Dhrumil-Sentry]

@ve-varun-sharma There's nothing beyond our public APIs: https://docs.sentry.io/api/

I'd love to understand your reason to export this data - Do you find the dashboards limiting in Sentry?

[peterHakio]

@Dhrumil-Sentry not being @ve-varun-sharma

1. We find it limiting, both in terms of functionality and with rate limiting
2. We already have bigQuery dashboard, so it is nice to have things there and there is much more to learn. Its nicer to be able to reuse some of the things we are allready doing

In my case it could be solve if I could get data viewer access to my own dataset in bigQuery. Suggestion explained here https://github.com/getsentry/sentry/issues/83222

[ale-cota]

hi @peterHakio,

We find it limiting, both in terms of functionality and with rate limiting

Can you please elaborate and let us know more about the limitations you're struggling with in Sentry dashboards? Thank you.

[rohit-gohri]

let us know more about the limitations you're struggling with in Sentry dashboards

(not peter, but sharing my usecase)

The biggest limitation I find is lack of variables (similar to what Grafana has, https://grafana.com/docs/grafana/latest/dashboards/variables/). This means I have to create the same dashboard multiple times or copy the same widget multiple times with slightly different set of filters but what I want is to quickly switch b/w different subset of filters through a single global filter.

And not like the current environment/project filters available because sometimes I do want a widget with total aggregated data, along with environment specific ones to see which environment has an anomaly compared to overall trends.

[IchordeDionysos]

Things like analysing crashes per team / feature based on the stacktrace would be a use case we are looking at. Especially when the team information is stored outside of Sentry this is kinda necessary to extend the default behaviour.

[sentaur-athena]

Building data export integrations are not on the roadmap now. So I pass it to @ale-cota and team to see how we can improve dashboards product to answer the needs.