Add a way to log and view metrics

[RichiCoder1]

Currently considering Exceptionless and the one big missing thing we'd love is the ability to track metrics, specifically performance metrics. An application could be basically not working and still never throw and exception. Things like request times, query times, performance counters, contention, and high levels of requests can be extremely valuable information to have for diagnosing issues.

First class support for these would be awesome, but failing that, a way to send data to Exceptionless in a https://github.com/exceptionless/Foundatio#metrics like way and get good visualizations and alerts for it would be great.

[niemyjski]

Yes, currently we use statsd/librato for application stats for exceptionless and really like it. It would be nice to have a dashboard with some basic gauges / stats for this and alerts when we don't get a specific event in for a while at some point. Since we do have feature usage it might be nice to have usage times displayed. I'm not sure if this is out side of the scope.

@ejsmith @RichiCoder1 thoughts?

[RichiCoder1]

While it's outside of the scope of the name "Exceptionless", I think it's very in scope as a solution for diagnosing apps in production. Just about every app monitoring solution (new relic, app insights, etc...) provides some level of support for logging quantity, duration, or percentage of some metric. Even basic support would be fantastic.

[RichiCoder1]

I should give some back story too as to why this is valuable. The apps we construct are largely for internal use only. LoB apps. Because of this, more than a few don't either A) allow access to the general internet or B) don't want data (even anonymized) being transmitted to third party services. For the ones that have neither restriction, we'd normal go with on the above providers I mentioned. But because we can't go to the cloud, we need good hosted solutions of which there aren't many. Enter Exceptionless. Meets our logging and exception needs very well, but we still need some place to send metrics. Short of a Metrics.Net/Kibana nightmare, there aren't lot of good solutions. So there's a lot of value in extending Exceptionless to make it the one stop shop for this need.

[niemyjski]

Yeah I agree with you. We'll keep this in mind. We are currently working on site performance and then will be focusing on feature work.

[RichiCoder1]

Sounds good! Let me know if and how I can help move this along :)

[ejsmith]

I'm worried about us adding more features vs improving the ones that we do have, but maybe there is something simple that we can do. How would you see this working?

[RichiCoder1]

Probably the simplest, for a given level of "simple", would be to allow submitting various metrics and then viewing them in an app sights-esque configurable dashboard: https://azure.microsoft.com/en-us/documentation/articles/app-insights-overview-usage/

[ejsmith]

If you just want counters, then we basically give you that ability now with being able to report events and then chart occurrences of those events over time. But timings are a whole new beast.

We are completely based around events right now... so we would have to have the concept of timings probably with percentiles and then be able to chart them over a filtered set of events that may or may not be timings. So most likely we would need a separate store just for timings which would introduce a whole new level of complexity to the entire app and clients as well.

I would love to have this feature as well, but I think the project needs to try and stay focused on doing events well for now.

[ejsmith]

I have been thinking about this some more and I think I have some ideas on how we can do it easily.

• Add new event types for counter, gauge, and timing
  • Doesn’t really change anything on the backend, we already have a value field on events and we would just populate that field.
  • Site menu currently has all of our event types as 1st class menus, but this would be too many. So we need to think about how the navigation might change.
  • Add helper methods to the client to report these types of events.
  • We could pre-aggregate the data before sending a batch of events. Similar to how statsd does. Basically just summing the counter values into a single event. (Can happen later as it just saves us some processing work)
  • How are feature usage and counter different?
• Add ability to create a custom reports page
  • Possibly use this: http://pkukielka.github.io/angular-panels/
  • Can specify the name of the report and would be listed under the reports section in the menu
  • 2 types of widgets: timeline chart and number
    • Timeline would allow you to add multiple event stacks to the chart
    • When adding a stack, you can pick occurrences, value sum and value average.
    • You can label each stack you add we show that label in the chart / hover
    • You can select the color for the line on the chart
    • You can provide a value type description (like seconds or milliseconds or count)
    • You can provide a value transform function to do something like convert from milliseconds to seconds. This could happen after the initial launch.
  • Number would allow you to pick a single stack and would show the event value from the most recent event.
    • Would be able to add a label to the widget header

[matteomigliore]

I report the request done to Blake to get:

• most day of week with errors
• most hour of the day with errors
• most used operating system
• most used browser

[rushfrisby]

I was just about to open an issue for this. Performance metrics would make exceptionless a good competitor to app insights on azure. I think you would need to adjust the pricing/number of events though - longer retention in exceptionless but more events in App insights. I'm unsure of what Session data vs Other data points is on azure though so maybe not.

[niemyjski]

You can do this today by submitting an event and specifying a value on the event. On the stack page you can see the value by sum, average or count. But it would be nice to have this as a first class citizen