aws-lambda-stream icon indicating copy to clipboard operation
aws-lambda-stream copied to clipboard

add metrics support

Open jgilbert01 opened this issue 5 years ago • 2 comments

  • size of batch
  • pipeline start and end timer
  • eventType ratio
  • other
  • cw, datadog, other
  • toPromise2 that returns metrics
  • dynamodb capacity units?
  • enable/disable via global/local options ?

jgilbert01 avatar Dec 04 '19 19:12 jgilbert01

In addition to what you listed above:

  • Lambda Xray integration
  • New relic?
  • Memory cache hits / misses
  • Timing data - beyond just start and end timers for the pipeline, I'm thinking about enabling performance profiling. A utility you wrap pipeline steps in that adds additional timing data at a more granular level
  • Pipeline ratios for batch - More so than just eventTypes since in practice a single event type might get processed by multiple pipelines

petermyers avatar Mar 08 '24 15:03 petermyers

  • defines a standard "metrics" field on uow
  • sinks/queries/steps/etc will add metrics as appropriate to the uow
  • a new toPromiseWithMetrics which will create histograms from uow.metrics per pipeline with min/avg/max/etc
  • and write the metrics as cloudwatch logs embedded metrics
  • https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html
  • return subset of metrics instead of Success

jgilbert01 avatar Jun 06 '24 04:06 jgilbert01

Can we call this issue resolved at this point?

petermyers avatar Sep 29 '24 14:09 petermyers