Add error reporting

[chadwhitacre]

Duplicates

• [X] I have searched the existing issues

Summary 💡

I had a call this morning with @Pwuts and @ntindle. They reached out to ask about adding error reporting using Sentry (disclosure: my employer). Sentry offers free SaaS for open source projects, though there are volume limits (working to determine these, will follow up with details when I have them).

The reason to add error reporting is to be able to find out about bugs in the software without having to wait for users to report them manually. This is a key enabler of shipping high-quality software.

Rough sketch:

1. Sign up on https://sentry.io/for/open-source/.
2. Add a consent prompt. Prompt once on first run then save in env (will require some refactoring). Consent should be broad to allow for additional info collection in future (e.g., performance data) without re-prompting. Here's an example from Sentry's own self-hosted installer.
3. How do we prevent or distinguish reports from forks vs. reports from the main project? App already checks for stable branch on startup, can extend to check repo. What about release artifacts though? Much less likely to be coming from a fork in a release artifact, so all release artifacts can send.
4. Add Sentry to existing error handling (example). Unhandled exceptions should be reported to Sentry by default out of the box.
5. Add logfile attachments, observing attachment size limits (tail -c? compress?).
6. Return a URL or ID for users to use when opening up bug reports, to tie their report to the report in Sentry.
7. Breadcrumbs?
8. Releases?

Examples 🌈

Here's an example from Sentry's own self-hosted installer.

Motivation 🔦

The reason to add error reporting is to be able to find out about bugs in the software without having to wait for users to report them manually. This is a key enabler of shipping high-quality software.

[chadwhitacre]

Sponsored plan details:

5M errors / 100M transactions / 10GB attachments / 100K replays Business features No term limit Can enable on-demand budget

[jerrodcodecov]

Thanks @chadwhitacre -- I think this would be awesome if we could get it working.

This would also enable code coverage in the stack trace as well: https://blog.sentry.io/2023/03/29/code-coverage-insights-now-in-your-stack-trace-announcing-our-codecov/

[lc0rp]

@p-i- Able to help with step 1?

[bfalans]

@chadwhitacre would you be available for a meeting on this somewhere next week?

[chadwhitacre]

Sure. Send me an email [email protected] and we can schedule. I'll aim to respond to your email on Monday morning.

[chadwhitacre]

Okay the significant-gravitas Sentry org should now be on a sponsored open source plan. Lmk if not. Have at it! :)

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions[bot]]

This issue has automatically been marked as stale because it has not had any activity in the last 50 days. You can unstale it by commenting or removing the label. Otherwise, this issue will be closed in 10 days.

[github-actions[bot]]

This issue was closed automatically because it has been stale for 10 days with no activity.

[zedatrix]

This would be a super cool feature

[Pwuts]

We're reprioritizing this. TODO:

• [x] Find and update GitHub issue about Sentry integration
• [x] Find out who is admin of the significant-gravitas org
• [x] Code
  • [x] Sentry integration
  • [x] Telemetry opt-in flow or config attribute

[chadwhitacre]

Yay! 😍 💃

[chadwhitacre]

393d6b97e6fc56995bb1753b0c976d91cf449f98 for the record. 👍

Cheers! 🍻 🙂

[jerrodcodecov]

@Pwuts please give us feedback if you have it :)