opentelemetry-collector-contrib ClickHouse - ScopeAttributes aren't stored

Component(s)

No response

What happened?

Description

Hi, I found out that scope attributes attached to a Meter aren't forwarded to ClickHouse. It seems, they aren't passed here https://github.com/wundergraph/opentelemetry-collector-contrib/blob/dbb685a63b0527c1255663e11daba27ca94e60dc/exporter/clickhouseexporter/internal/histogram_metrics.go#L178

Steps to Reproduce

Create a meter with the following option and measure anything.

routerMeter := h.meterProvider.Meter(cosmoRouterMeterName,
	metric.WithInstrumentationVersion("0.0.1"),
	metric.WithInstrumentationAttributes(attribute.String("router.version", "0.0.1")),
)

Expected Result

I should see attributes in the ScopeAttributes column.

Actual Result

No attributes.

Collector version

0.84.0

Environment information

Environment

OS: (e.g., "Ubuntu 20.04") Pop!_OS 22.04 LTS x86_64 Compiler(if manually compiled): (e.g., "go 14.2") go version go1.20.4 linux/amd64

OpenTelemetry Collector configuration

receivers:
  otlp:
    protocols:
      http:
        endpoint: '0.0.0.0:4318'

exporters:
  clickhouse:
    endpoint: ${CLICKHOUSE_ENDPOINT}
    timeout: 10s
    database: cosmo
    traces_table_name: otel_traces
    metrics_table_name: otel_metrics
    ttl_days: 30
    sending_queue:
      queue_size: 5000
    retry_on_failure:
      enabled: true
      initial_interval: 10s
      max_interval: 30s
      max_elapsed_time: 300s

service:
  pipelines:
    metrics:
      receivers: [otlp]
      exporters: [clickhouse]
    traces:
      receivers: [otlp]
      exporters: [clickhouse]

Log output

No response

Additional context

No response

Oct 04 '23 11:10 StarpTech

Pinging code owners for exporter/clickhouse: @hanjm @dmitryax @Frapschen. See Adding Labels via Comments if you do not have permissions to add labels yourself.

Oct 04 '23 16:10 github-actions[bot]

I don't have much context here, but it looks like the scope attributes are actually getting passed in where you've referenced. The scopeInstr variable is the instrumentation scope, which includes a getter method Attributes() (the scope instrumentation attributes). From the flow of metrics in this exporter, it looks like after all the histogram metrics are all added to a list together, they get inserted into Clickhouse using the insert method. From here we find the instrumentation scope attributes are converted into a key-value pair map, and then theoretically inserted into the DB using the SQL command. The SQL insert command being executed is found here.

I'll have to defer to code owners to confirm what I've found. They might have some more context as to why things are different than expected here.

Oct 05 '23 16:10 crobert-1

@StarpTech Can you make the collector config shorter, so that it is easier for others to reproduce?

Oct 10 '23 10:10 cwegener

Done

Oct 10 '23 10:10 StarpTech

@StarpTech Does this issue still remain or should I close it?

Oct 16 '23 09:10 Frapschen

Hi, I can still reproduce it.

Oct 16 '23 12:10 StarpTech

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/clickhouse: @hanjm @dmitryax @Frapschen

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Dec 18 '23 03:12 github-actions[bot]

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/clickhouse: @hanjm @dmitryax @Frapschen

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Feb 19 '24 03:02 github-actions[bot]

@Frapschen Can the code owners provide any feedback?

Feb 19 '24 10:02 StarpTech

@StarpTech The integration test(https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30209) has verified that the scope data has been sent to clickhouse: https://github.com/open-telemetry/opentelemetry-collector-contrib/pull/30209/files#diff-1ec84547ed93f2c8bfb21c371ca0b5304f01371e748d4b02bf397313a4b1dfa4R406-R412

Mar 14 '24 09:03 Frapschen

I'll try to reproduce it again and come back.

Mar 14 '24 10:03 StarpTech

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/clickhouse: @hanjm @dmitryax @Frapschen @SpencerTorres

See Adding Labels via Comments if you do not have permissions to add labels yourself.

May 14 '24 03:05 github-actions[bot]

Hey @StarpTech! Were you able to reproduce this?

Jun 24 '24 18:06 SpencerTorres

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/clickhouse: @hanjm @dmitryax @Frapschen @SpencerTorres

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Aug 26 '24 03:08 github-actions[bot]

@StarpTech let me know if this is still an issue. Thanks!

Aug 26 '24 18:08 SpencerTorres

This issue has been inactive for 60 days. It will be closed in 60 days if there is no activity. To ping code owners by adding a component label, see Adding Labels via Comments, or if you are unsure of which component this issue relates to, please ping @open-telemetry/collector-contrib-triagers. If this issue is still relevant, please ping the code owners or leave a comment explaining why it is still relevant. Otherwise, please close it.

Pinging code owners:

exporter/clickhouse: @hanjm @dmitryax @Frapschen @SpencerTorres

See Adding Labels via Comments if you do not have permissions to add labels yourself.

Oct 28 '24 03:10 github-actions[bot]

This issue has been closed as inactive because it has been stale for 120 days with no activity.

Dec 27 '24 05:12 github-actions[bot]