feature: Need Cardinality Limiter for gql.loaderBatchSize tag in gql.dataLoader metrics

[hzariv]

Please read our contributor guide before creating an issue. Also consider discussing your idea on the discussion forum first.

Describe the Feature Request

gql.loaderBatchSize tag in gql.dataLoader metrics can have a high cordinality. I need a way to limit this tag cardinality.

Describe Preferred Solution

I would like management.metrics.dgs-graphql.tags.limiter.limit property to apply to gql.loaderBatchSize tag

Describe Alternatives

Provide a way for me to customize the metrics tags

[mrvaruntandon]

Hi @hzariv,

Is there any discussion thread on this feature request that I can read to understand this issue better? if no, could you please explain the request a little more?

[hzariv]

@mrvaruntandon We have many applications that use @DgsDataLoader heavily with diffrent batch sizes. This results in high cardinality in our metrics system. Here is an example of these metrics. As you can see gql_loaderBatchSize is unbounded and can very. This results in cardinality issues for us. So I need a way to limit the cardinality for this tag or a way to customise the gql_dataLoader_seconds_count and gql_dataLoader_seconds_sum metrics.

gql_dataLoader_seconds_count{gql_loaderBatchSize="152",gql_loaderName="Extension",} 5.0 gql_dataLoader_seconds_sum{gql_loaderBatchSize="152",gql_loaderName="Extension",} 0.286892171 gql_dataLoader_seconds_count{gql_loaderBatchSize="140",gql_loaderName="Extension",} 1.0 gql_dataLoader_seconds_sum{gql_loaderBatchSize="140",gql_loaderName="Extension",} 0.061202343 gql_dataLoader_seconds_count{gql_loaderBatchSize="10",gql_loaderName="Extension",} 119.0 gql_dataLoader_seconds_sum{gql_loaderBatchSize="10",gql_loaderName="Extension",} 0.442741418 gql_dataLoader_seconds_count{gql_loaderBatchSize="176",gql_loaderName="Extension",} 4.0 gql_dataLoader_seconds_sum{gql_loaderBatchSize="176",gql_loaderName="Extension",} 0.299238463

[mrvaruntandon]

@hzariv, Thank you for sharing the behaviour you are seeing.

Do you have some sample project from which I can replicate this high cardinality behaviour explained in this issue?

[hzariv]

@mrvaruntandon the following example project already has a MappedBatchLoader so if the query is asking for a group of shows with multiple reviews you will get the dataloader metrics as describe here with loaderBatchSize tag in dataLoader metrics. There is nothing special here that application is using. If the caller send queries for multiple shows with a number of reviews, DGS framework will create dataLoader metrics with different batch sizes and that can result in high cardinality.

https://github.com/Netflix/dgs-examples-java/blob/main/src/main/java/com/example/demo/dataloaders/ReviewsDataLoader.java

[mrvaruntandon]

@hzariv are you still facing this issue?

[hzariv]

@mrvaruntandon I ended up removing gql.loaderBatchSize tag from gql.dataLoader.seconds* metrics using a MeterFilter to circumvent the problem.

[srinivasankavitha]

@hzariv - how did you install the MeterFilter? Per this discussion here, you mentioned you couldn't accomplish that because the MeterRegistry bean cannot be customized because DgsMeterRegistrySupplier is internal: https://github.com/Netflix/dgs-framework/discussions/1986#discussioncomment-10330913