opentelemetry directives unknown

[BrendanGalloway]

Describe your environment

Running nginx in a kubernetes pod to serve static content and proxy dynamic requests.

Steps to reproduce

Nginx pod was build with this dockerfile:

# Take an upstream docker image as an argument
# build the static directory there

# SRC_IMG will be the `web` container, with the assets collected in /app/static-collected
ARG SRC_IMG

FROM ${SRC_IMG} AS static_source

# When updating nginx also make sure to check otel module compatibility matrix and update otel version accordingly
FROM nginx:1.23.1
ADD https://github.com/open-telemetry/opentelemetry-cpp-contrib/releases/download/webserver%2Fv1.0.3/opentelemetry-webserver-sdk-x64-linux.tgz /opt
RUN cd /opt; tar xvfz opentelemetry-webserver-sdk-x64-linux.tgz
RUN cd /opt/opentelemetry-webserver-sdk; ./install.sh
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/opentelemetry-webserver-sdk/sdk_lib/lib
RUN mkdir /app/
COPY --from=static_source /app/static-collected /app/static/

adding in static content is probably not necessary to reproduce.

nginx.conf, otel-nginx.toml and also opentelemetry_module.conf are loaded into the pod via configmap:

data:
  nginx.template.conf: |-
    load_module /opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/${NGINX_VERSION}/ngx_http_opentelemetry_module.so;

    user www-data;
    worker_processes 8;
    error_log  /var/log/nginx/error.log warn;
    pid        /var/run/nginx.pid;

    # https://serverfault.com/questions/787919/optimal-value-for-nginx-worker-connections
    events {
      worker_connections 10240;
    }

    http {
      opentelemetry_config /conf/otel-nginx.toml;
      
      upstream varnish_upstream {
        server ${VARNISH_HOST}:8040;
        keepalive 32;
      }

      server {
        listen 80 default_server;
        listen [::]:80;
        server_name _; 

        location / {
            proxy_send_timeout  300;
            proxy_read_timeout  300;
            proxy_set_header Host $host;
            proxy_set_header X-Real-IP $remote_addr;
            proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
            proxy_set_header X-Forwarded-Proto https;
            proxy_set_header X-Forwarded-Port 443;
            proxy_pass http://varnish_upstream;
        }

      } # server
    } # http
  opentelemetry_module.conf: |-
    NginxModuleEnabled ON;
    NginxModuleOtelSpanExporter otlp;
    NginxModuleOtelExporterEndpoint http://otel-collector-collector.monitoring:4317;
    NginxModuleServiceName nginx-instrumentation;
    NginxModuleServiceNamespace default;
    NginxModuleServiceInstanceId nginx-instrumentation-test;
    NginxModuleResolveBackends ON;
    NginxModuleTraceAsError OFF;
  otel-nginx.toml: |-
    exporter = "otlp"
    processor = "batch"

    [exporters.otlp]

    host = "otel-collector-collector.monitoring.svc.cluster.local"
    port = 4317

    [processors.batch]
    max_queue_size = 2048
    schedule_delay_millis = 5000
    max_export_batch_size = 512

    [service]
    name = instrumentation-nginx

and the nginx.conf file is templated with the varnish and nginx version values before nginx starts.

What is the expected behavior? Nginx starts correctly and sends traces to the otel collector

What is the actual behavior?

nginx crashes with the message:

nginx: [emerg] unknown directive "opentelemetry_config" in /nginx.conf:14

Additional context

I have also tried using the instructions in the opentelemetry blog which configure the module using include /etc/nginx/config.d/*.conf instead of the opentelemetry_config directive. In that case nginx starts up and I see the error I describe in !255. Traces appear in Jaeger, but seemingly without propagation configured. If I try add opentelemetry directives, such as opentelemetry_propogation or opentelemetry_trust_upstream_traces, nginx crashes as described above

[gnd]

Hello, what is the status of this issue ?

I have the same problem with Nginx 1.22.0 and OpenTelemetry Nginx module v1.0.3. It doesnt recognize the opentelemetry_* instructions:

unknown directive "opentelemetry_config" in /etc/nginx/nginx.conf:73

[nise-wg2]

I am facing the same issue. Followed https://github.com/open-telemetry/opentelemetry-cpp-contrib/blob/main/instrumentation/nginx/README.md.

Neither opentelemetry_propagate nor opentelemetry_attribute are recognized.

OpenTelemetry module is otherwise working and sending traces to the oltp enpoints for collection.

[43034r]

Same issue with "unknown directive" opentelemetry_operation_name _; opentelemetry_propagate; https://opentelemetry.io/blog/2022/instrument-nginx/

[PandeoF1]

Same issue with "unknown directive"

[PandeoF1]

Update: I have found the nginx official module that are more optimized (they say that not me) And it work very well : http://nginx.org/en/docs/ngx_otel_module.html