setting index labels with `otlp_config` does not work

[rooffl]

Describe the bug I think it's similar to https://github.com/grafana/loki/issues/12780 but even with the fix for this it seems my issue persists. I'm pushing logs to Loki with the otel collector. Currently I'm using the lokiexporter but since Loki supports a native OTLP endpoint now I'd like to switch to using the otlphttp exporter and configure Loki to set the index labels i used previously. Previously I achieved this by using the attributes processor with the config (where foo is the attribute i want to set as an index label):

attributes/loki-labels:
  - action: insert
    key: loki.attribute.labels
    value: foo

I elevated my attributes (here foo as an example) to resource attribute level since from my understanding, setting attributes as index_labels is only possible for resource attributes. My limits_config for Loki is this:

limits_config:
  otlp_config:
    resource_attributes:
      ignore_defaults: false
      attributes_config:
        - action: index_label
          attributes:
            - foo

and the attribute shows up as a Field but so do attributes not specified in the config. However it does not show up in the Label filters dropdown on the Explore page and I also can't query my logs with {foo="bar"}.

This indicates to me that it's not set as index/log label.

Here is an example log record:

ResourceLog #1
Resource SchemaURL: 
Resource attributes:
     -> container.image: Str(REDACTED)
     -> host.name: Str(REDACTED)
     -> machine.id: Str(REDACTED)
     -> origin: Str(REDACTED)
     -> foo: Str(bar)
ScopeLogs #0
ScopeLogs SchemaURL: 
InstrumentationScope  
LogRecord #0
ObservedTimestamp: 2024-07-08 09:06:57.441277509 +0000 UTC
Timestamp: 2024-07-08 09:06:57.33826 +0000 UTC
SeverityText: 
SeverityNumber: Unspecified(0)
Body: Map(REDACTED)
Trace ID: 
Span ID: 

If I'm misunderstanding or missing something please correct me.

Expected behavior I expected the attributes specified to be applied as index labels and show up in the drop down/ be able to query my logs with {foo="bar"}.

Environment:

• Infrastructure: Fedora Linux 38
• Deployment tool: docker
• Loki version 3.1.0

[pr1337]

Since foo is part of Resource attributes as shown in your example log record, I think you should be able to index it by configuring the distributor:

loki:
  distributor:
    otlp_config:
      default_resource_attributes_as_index_labels:
        - foo

This will override the default list, so if you want to keep any of the other values, you have to add them too. You can check the default values here: https://grafana.com/docs/loki/latest/send-data/otel/#format-considerations

[ghost]

Since foo is part of Resource attributes as shown in your example log record, I think you should be able to index it by configuring the distributor:

loki:
  distributor:
    otlp_config:
      default_resource_attributes_as_index_labels:
        - foo

This will override the default list, so if you want to keep any of the other values, you have to add them too. You can check the default values here: https://grafana.com/docs/loki/latest/send-data/otel/#format-considerations

I see! I must've missed the option to configure it in the distributor directly. Thank you! Just to clarify for my understanding: I take it the config in the distributor sets it globally vs a per tenant config in the limits_config block?

[Vinaum8]

+1

docker.io/grafana/loki:3.1.1 helm chart version:

component: loki
project: default
repoURL: https://grafana.github.io/helm-charts
version: 6.10.0

[Vinaum8]

in this link, Travis proposal one config, but i think what not a good aproach.

[evilr00t]

Bump - it looks like a bug as only host.name cannot be indexed, didn't notice issues with other attributes.