posthog icon indicating copy to clipboard operation
posthog copied to clipboard

Threshold-based alerting

Open annikaschmid opened this issue 2 years ago • 7 comments

Users have asked for a functionality to create threshold-based alerts, e.g. when the conversion rate drops below a certain threshold.

Open questions:

  • In what format do people want these alerts? Email? Slack?

We'd love to hear user feedback, as well as ideas for stretch goals or implementation.

If you like this idea, please leave a 👍 or ❤️ reaction on this post to vote for it -- your votes and feedback help us prioritize what to work on next!

annikaschmid avatar Feb 21 '23 11:02 annikaschmid

Notification UI as well possibly?

voarsh2 avatar Mar 05 '23 04:03 voarsh2

I'd find this useful! Both for conversion rates and overall totals (if errors today > X, if visitors/hour < Y).

As a stretch goal, it'd be useful to be able to alert on major changes in breakdowns. For example, I have a graph showing traffic to my site by day broken down by referrer - if there's a major change in who's referring traffic (a bunch of new traffic from a new source, or a previous major source of traffic disappearing) I'd love to be able to get alerts about that, even if the overall traffic total doesn't shift. Harder to define I imagine, but I think there's value in there.

In what format do people want these alerts? Email? Slack?

Personally: primarily email, and then webhooks. Email is quick & easy to get started, and medium-term webhooks would be the best way to integrate this into the rest of my stack (slack, pagerduty, internal dashboarding, "reboot the server automatically if load time gets too high", whatever).

pimterry avatar Apr 20 '23 09:04 pimterry

Slack alert based on insight threshold triggers 👍

nik-418 avatar Jul 06 '23 16:07 nik-418

I need this. Other product analytics platforms like Mixpanel and Amplitude offer this. There's no reason Posthog also shouldn't. Please prioritize this.

mindofmar avatar Aug 07 '23 17:08 mindofmar

Consider a mobile companion app that sends push notifications as well, see request.

marandaneto avatar Feb 05 '24 09:02 marandaneto

Same as the alert per action - alerts per insight by threshold.

GalDayan avatar Mar 27 '24 14:03 GalDayan

+1, but offer email + slack please. We use discord instead of slack, so email notifications would be great.

williamlmao avatar May 24 '24 18:05 williamlmao

Basically this would give observability capabilities to PostHog. One can use the platform not only for product analytics, but also as alerting tool if some of the metrics go down or up too rapidly. Highly anticipated feature!

The way how notifications/alerts are delivered is of less importance, it can be a simple email.

Slawikus avatar Jun 21 '24 11:06 Slawikus

This would be useful as a simple canary to catch misconfigured deployments - one could (hypothetically) introduce a regression where you stop sending events to Posthog (I, however, would never do such a thing 😅). An alert on a sudden decrease in any metrics would catch this.

blackbird-chrisf avatar Jul 23 '24 14:07 blackbird-chrisf

We've started work on this feature. Will have updates soon on how users can sign up for early access to this feature!

anirudhpillai avatar Sep 23 '24 15:09 anirudhpillai

awesome! email alerts for me, same as google analytics insights stuff

stuartbrameld avatar Sep 30 '24 15:09 stuartbrameld

Big +1 to this, having this feature would save us a bunch of work we otherwise have committed to do soon on KPI alerts 🙏

Bogdanovist avatar Oct 02 '24 01:10 Bogdanovist

We have a very rough version of alerting available for trends without breakdowns (breakdown support coming soon) on which we wanted some user feedback. Have enabled a few people for this feature, if anyone's very keen try it out (or submit feedback) just reach out to me at [email protected]

The current version, lets you set alerts on trends without breakdowns from the dropdown on the page for the insight, and you can view all the alerts you've setup on the 'Product Analytics' page under the 'Alerts' tab

image image

anirudhpillai avatar Oct 03 '24 10:10 anirudhpillai

+1 https://posthoghelp.zendesk.com/agent/tickets/18810

abigailbramble avatar Oct 03 '24 15:10 abigailbramble

Just an update, we now support setting alerts for monitoring day over day/week over week change in an insight value (just as requested by a lot of users)! You can also set these 'relative' alerts to trigger for percentage based thresholds (eg. signups decreasing by over 5% week over week)

image image

Alerts which are firing also support a 'snooze button' now. So you can choose not to get notified about the alert until a preset duration.

image

Feel free to send any feedback/requests to me directly at [email protected] :)

anirudhpillai avatar Oct 21 '24 12:10 anirudhpillai

You can also set these 'relative' alerts to trigger for percentage based thresholds (eg. signups decreasing by over 5% week over week)

nice 👍

stuartbrameld avatar Oct 21 '24 18:10 stuartbrameld

Also for alerts comparing absolute value of the insight (not 'relative', day over day/week over week), you can choose to display the threshold lines on the insight. To make this persist, you'll have to 'Edit' the insight > 'Options' > 'Show alert threshold lines'

image

anirudhpillai avatar Oct 23 '24 07:10 anirudhpillai

Would it be the right feature for Web vitals thresholds or is something else in the works? I don't exactly need alerts or to be notified when something crosses a given threshold but I would be interested in seeing the "good", "needs improvement" and "poor" thresholds as colored backgrounds in my charts.

angrybacon avatar Oct 30 '24 14:10 angrybacon

Hey @angrybacon, I made a separate issue for this here https://github.com/PostHog/posthog/issues/26031

robbie-c avatar Nov 06 '24 17:11 robbie-c

@robbie-c I'm confused, how would that help seeing hardcoded threshold areas in charts? I mentioned Web vitals because that's what my use case is but there could be more so I'm assuming we want "hardcoded threshold areas" to be metric-agnostic, if that makes sense.

angrybacon avatar Nov 06 '24 17:11 angrybacon

We need alerting for funnels, so that we can be alerted to changes in conversion rates. Please like this 👍 if alerting for conversation rates in funnels would be valuable to you. Thanks!

willzoltan avatar Nov 24 '24 21:11 willzoltan

Thanks Will, agreed that's very useful so we'll be adding support for alerts on funnels soon!

In other news, now we also support setting alerts on trends with breakdowns! These alert when any breakdown value breaches the thresholds set.

image

anirudhpillai avatar Nov 28 '24 14:11 anirudhpillai

Closing this issue as we now have insight alerts in PostHog. Feel free to open separate issues for further enhancements to this, it will help us prioritise them correctly. I've already logged #27300 and #27302 which are both highly requested.

anirudhpillai avatar Jan 07 '25 10:01 anirudhpillai