gitpod icon indicating copy to clipboard operation
gitpod copied to clipboard

Stripe: Create observability into the type, rate and return code of API calls we issue against Stripe

Open geropl opened this issue 3 years ago • 4 comments

As it says in the title. An example is the metric gitpod_server_api_calls_total (link) from which we get insight into error rates, repsonse times and type of method called.

geropl avatar Sep 21 '22 08:09 geropl

Implementation-wise (for Golang), we should use a HTTP Client tripper to compose an HTTP client which also exports metrics. See https://dev.to/stevenacoffman/tripperwares-http-client-middleware-chaining-roundtrippers-3o00 for examples

easyCZ avatar Sep 21 '22 08:09 easyCZ

FYI, Stripe Dashboard > Developers already has this:

Screenshot 2022-09-21 at 10 58 00 Screenshot 2022-09-21 at 10 58 11

And you can drill down into the specific error codes:

Screenshot 2022-09-21 at 11 01 53

jankeromnes avatar Sep 21 '22 08:09 jankeromnes

Still seems worth implement the low-hanging parts on our end. a) Grafana is more flexible in displaying data, and b) we can build alerts with our common tools.

geropl avatar Sep 21 '22 09:09 geropl

Still seems worth implement the low-hanging parts on our end. a) Grafana is more flexible in displaying data, and b) we can build alerts with our common tools.

Stripe will only have server metrics - for requests that actually reached it. If we fail to contact Stripe for whichever reason, we'd be blind - we need client side metrics also.

easyCZ avatar Sep 21 '22 09:09 easyCZ

Client-side metrics from server PR is here

easyCZ avatar Oct 06 '22 11:10 easyCZ

Metrics from server are coming through nicely on staging

Image

However, metrics from usage include the full path in their labels which also includes Subscription IDs - this would lead to high cardinality of these metrics and will need to be reworked. We'll need to strip the IDs and sub-resources. Metrics reported from usage

easyCZ avatar Oct 06 '22 19:10 easyCZ

Fix for the high cardinality metrics is here - https://github.com/gitpod-io/gitpod/pull/13701

easyCZ avatar Oct 10 '22 08:10 easyCZ

Metrics are now exposed properly see dashboard

easyCZ avatar Oct 12 '22 11:10 easyCZ