loki
loki copied to clipboard
Log lines with same timestamp are not in order after ingesting to Loki
Describe the bug I am ingesting the log files to Loki via Fluent-bit loki output plugin, the log lines with same timestamp were not in the order as they were in the original logs.
Original log files:
[D 2024-06-19 17:25:02.417 26256 17660 - - ldap3:118] BASIC:instantiated Tls: <Tls(validate=<VerifyMode.CERT_NONE: 0>)> [D 2024-06-19 17:25:02.417 26256 17660 - - ldap3:118] BASIC:instantiated Server: <Server(host=‘shgmwcdc01.cod.ms.com.cn’, port=636, use_ssl=True, allowed_referral_hosts=[(‘’, True)], tls=Tls(validate=<VerifyMode.CERT_NONE: 0>), get_info=‘SCHEMA’, mode=‘IP_V6_PREFERRED’)> [D 2024-06-19 17:25:02.417 26256 17660 - - ldap3:118] ERROR:unknown strategy for <ldaps://shgmwcdc01.cod.ms.com.cn:636 - ssl - user: itsm_api_qa - not lazy - unbound - closed - - tls not started - not listening - No strategy - internal decoder> [D 2024-06-19 17:25:02.417 26256 17660 - - ldap3:118] BASIC:instantiated : <ldaps://shgmwcdc01.cod.ms.com.cn:636 - ssl - user: itsm_api_qa - not lazy - unbound - closed - - tls not started - not listening - No strategy - internal decoder - async - real DSA - not pooled - cannot stream output> [D 2024-06-19 17:25:02.417 26256 17660 - - ldap3:118] BASIC:instantiated Connection: <Connection(server=Server(host=‘shgmwcdc01.cod.ms.com.cn’, port=636, use_ssl=True, allowed_referral_hosts=[(‘’, True)], tls=Tls(validate=<VerifyMode.CERT_NONE: 0>), get_info=‘SCHEMA’, mode=‘IP_V6_PREFERRED’), user=‘itsm_api_qa’, auto_bind=‘NO_TLS’, version=3, authentication=‘SIMPLE’, client_strategy=‘RESTARTABLE_ASYNC’, auto_referrals=True, check_names=True, read_only=False, lazy=False, raise_exceptions=False, fast_decoder=True, auto_range=True, receive_timeout=15, return_empty_attributes=True, auto_encode=True, auto_escape=True, use_referral_cache=False)>
Queried from Grafana:
Output with Fluent-bit stdout plugin:
Per the suggestion in cannot order loki entries with same timestamp · Issue #56071 · grafana/grafana · GitHub, I set increment_duplicate_timestamp
as true
, but still no luck.
To Reproduce Steps to reproduce the behavior:
- Started Loki
- Ingested log snippet above by retrieving the timestamp via regex
Expected behavior Log lines with same timestamp should be kept in order as they were in the original log files.
Environment:
- Infrastructure: K8S
- Deployment tool: helm
- Loki Version: 2.9.8
Screenshots, Promtail config, or terminal output Uploaded in above sections.