backend-best-practices icon indicating copy to clipboard operation
backend-best-practices copied to clipboard

Added a section for bad monitoring examples

Open laurianttila opened this issue 9 years ago • 5 comments

This was discussed in LCM related workshop, feel free to groom the text in case it's awful.

laurianttila avatar Jan 29 '16 06:01 laurianttila

A thought: since these "war stories" are a new thing, what do you think about putting them in a .md file or a section of their own, and then linking to them from a shorter text that has the basic problem description and recommendation? That way, in case the war stories include multiple examples of bad practices, they could be linked to from multiple places in the document. Maybe the stories themselves could even have a section that spells out the causes of the problems, and links to the relevant best practices.

ilkka avatar Feb 07 '16 08:02 ilkka

These kinds of war stories are really valuable! Would it make sense to link to a separate .md file from this document describing how things can do wrong, but then also add to this document solutions or proposals on how to do monitoring? Pingdom and New Relic kind of services come to mind, that can do "smart alerting" when something happens.

kypeli avatar Feb 07 '16 11:02 kypeli

war stories: write a blog post and link to it. Also discuss what should have been the right (or at least better) approach. I'm :-1: on merging this as is.

phadej avatar Feb 07 '16 16:02 phadej

Maybe rewrite this as "word of warning - use email alerts with caution" (with some additional words about hurry/small budget/easy way)? War stories then as a blog post :).

laurianttila avatar Feb 07 '16 18:02 laurianttila

I'd like to see something that clearly states that email alerts should be reserved for high priority, disruptive situations, and care should be taken to avoid false positives and to throttle them appropriately — otherwise they either get ignored, or cause more problems downstream with their volume.

su 7. helmikuuta 2016 klo 20.24 Lauri Anttila [email protected] kirjoitti:

Maybe rewrite this as "word of warning - use email alerts with caution" (with some additional words about hurry/small budget/easy way)? War stories then as a blog post :).

— Reply to this email directly or view it on GitHub https://github.com/futurice/backend-best-practices/pull/15#issuecomment-181073563 .

Ilkka Laukkanen Senior Specialist Futurice Oy

ilkka avatar Feb 08 '16 08:02 ilkka