Andrei Lalaev

Metrics labels and values in loki logs

21

Qryn Prometheus metrics duplicated in loki requests with labels and their values   it's not a problem when you can add a filter, such as `{job="loki", level=~""}` but if...

## Background: Qryn is a fantastic tool for monitoring and analyzing data, but it currently lacks support for multi-tenancy. Multi-tenancy is becoming increasingly important for organizations as they need to...

Slow metrics requests

6

Version: 3.0.26-bun (dockerhub) Promql request: `http_server_request_duration_seconds_bucket{job="Service", deployment_environment="Production", instance=~""}` request duration: 8 sec for 3 hours of metrics Clickhouse request: ``` SELECT count() AS series_n, sum(length(values)) AS points_n FROM ( select...

**Concisely describe the proposed feature** I propose adding functionality to integrate Active Directory for managing roles and group policies within the application. **Describe the solution you'd like (if any)** I...

[bug] RangeError: Too many properties to enumerate

3

│ {"level":50,"time":1712840081578,"pid":19,"hostname":"qryn-write-765d585cc6-vwjx5","name":"qryn","msg":"rawRequest error: INSERT INTO qrynprod.traces_input\n (trace_id, span_id, parent_id, name, timestamp_ns, duration_ns, service_name, payload_type, payload, tags) \n FORMAT JSONEachRow"} │ │ {"level":50,"time":1712840081580,"pid":19,"hostname":"qryn-write-765d585cc6-vwjx5","name":"qryn","reqId":"req-1o","err":"Database push error\nError: Database push error\n at Worker. (/app/lib/db/clickhouse.js:84:27)\n...

Tempo request raises out of bound error

2

Tempo request like {span.service.name = "svc" && span.name =~"api" && span.deployment.environment = "prod"} grafana return Query error Error (memory access out of bounds). Please check the server logs for more...

Traces insertion error

4

Sometimes with no reason I can see this error ``` qryn-write-6478699f66-bqh54 {"level":30,"time":1718178232495,"pid":19,"hostname":"qryn-write-6478699f66-bqh54","name":"qryn","reqId":"req-1jvh","r eq":{"method":"POST","url":"/v1/traces","hostname":"qryn-write.qryn.svc:3100","remoteAddress":"172.24.17.199","remotePort":51128},"msg":"incoming request"} qryn-write-6478699f66-7bbg9 {"level":50,"time":1718178232512,"pid":19,"hostname":"qryn-write-6478699f66-7bbg9","name":"qryn","reqId":"req-18is","e rr":"Cannot read properties of undefined (reading 'forEach')\nTypeError: Cannot read properties of undefined (reading 'forEach')\n...

Slow log request performance

13

log format: ``` {"body":"body","traceid":"1","spanid":"1","severity":"Information","flags":1,"attributes":{"AccountId":1,"ActionId":"UUID","ActionName":"Controller","Stat":"1","Id":"2","ConnectionId":"123","GId":0,"ParentId":"UUID","RequestId":"UUID","RequestPath":"/path","SpanId":"1323aff603f45021","TraceId":"03016d03700e03ec8b2892038bacfd92","{OriginalFormat}":"INIT"},"resources":{"deployment.environment":"Stage","service.instance.id":"UUID","service.name":"Service","team":"Team","telemetry.sdk.language":"dotnet","telemetry.sdk.name":"opentelemetry","telemetry.sdk.version":"1.9.0"},"instrumentation_scope":{"name":"Controller"}} ``` Loki request: {level=~"WARN|INFO|ERROR|FATAL", job="Service"} |= `85577686` | json body="body", Environment="resources[\"deployment.environment\"]", Scope="instrumentation_scope[\"name\"]" | Environment = `Stage` Took more then 30 seconds and crush then. `{level=~"WARN|INFO|ERROR|FATAL", job="Service"}...