Tracking request query parameters

[whatnick]

I recently replaced custom Prometheus metrics code in datacube-ows with this library. However most of the usage of this flask application is differentiated using get request query parameters as opposed paths or endpoints. What would be the relevant decorator / custom function to group by query parameters be.

Relevant code changes here : https://github.com/opendatacube/datacube-ows/pull/340

[rycus86]

Hi @whatnick

Have a look at a relevant test in test_group_by.GroupByTest.test_group_by_func. You can provide your own grouping function that groups requests by allowed query parameters for example (though I'd be cautious about potential cardinality there :) ).

def parameters(r):  # receives the Flask request object as argument
    return 'type_%s_operation_%s' % (r.args.get('type', 'unknown'), r.args.get('op', 'none'))

metrics = PrometheusMetrics(app, group_by=parameters)  # default metrics will have parameters=? labels instead of path=?

Another option is to use default labels that are added to every metric managed by the library:

from Flask import request

metrics = PrometheusMetrics(app, default_labels={
    'type': lambda: request.args.get('type', 'unknown'),
    'operation': lambda: request.args.get('op', 'none')
})

This one would keep the path=? label by default.

Does this help?

[whatnick]

Thanks for the response. Could this be moved to the docs for this project ? I will make a PR when I get time. What does cardinality here mean ? loss of order of query parameters ? The labels approach seems more interesting.

[rycus86]

See https://www.robustperception.io/cardinality-is-key about the cardinality problem. The tl;dr is that if you don't control the values you put in the metrics labels, your users can generate as many different values as thry want, and each metricname+labels combination will be an individual time series in Prometheus.

[whatnick]

Ah right it causes metric explosion since query parameters can be anything. Will attempt to restrict to a finite set during metrics generation.