filter out http_request_duration_seconds and http_requests_total?

[davidhu2000]

we have an API that look something like

/api/v1/blah/{id}/{type}

id - a number type - a string

with 2 dynamic variables. but the metrics coming back doesn't seem to parse the 2nd variable as a variable, but rather as a string

so we see stuff like

http_request_duration_seconds_bucket{le="0.1",route="/api/v1/blah/#val/type-1",method="GET",status="2XX"}
...
http_request_duration_seconds_bucket{le="0.1",route="/api/v1/blah/#val/type-2",method="GET",status="2XX"}

this causes the metrics API response time to skyrocket since there are tons of different types. Our payload size was a few MBs.

So I was wondering whether we can remove those particular metrics as we use other means to measure them.

IT does appear the #val replacement only happen for numbers. i tried locally to put type as number, and it is parsed as #val correctly.

[zbjornson]

I'm not following. Can you show the code you're using to define your histograms please?

[davidhu2000]

i'm not doing any custom configs, just this mainly

import express from 'express';
import promMid from 'express-prometheus-middleware';

const app = express();
app.set('port', process.env.PORT || 4000);
app.set('env', process.env.NODE_ENV);

app.use(express.json());
app.use(promMid({ metricsPath: '/prometheus' }));

I'm using https://www.npmjs.com/package/express-prometheus-middleware which relies on prom-client

essentially, when the route has two consecutive variables, the histogram does not parse correctly

/api/v1/blah/#val1/#val2

so I would expect the following two routes to be grouped together but metrics API is showing them as separate

/api/v1/blah/#val/type-1
/api/v1/blah/#val/type-2

[zbjornson]

Routes and variables are not concepts in prom-client itself, so this might be a bug in express-prometheus-middleware. Can you provide an example reproduction using only prom-client please?