opentelemetry-cpp-contrib icon indicating copy to clipboard operation
opentelemetry-cpp-contrib copied to clipboard

Alpine support?

Open davebullock-ujet opened this issue 1 year ago • 9 comments

Any plans on supporting alpine?

davebullock-ujet avatar Jul 21 '23 02:07 davebullock-ujet

@davebullock-ujet I am currently researching the same. We are using nginx:alpine image, and are able to build the docker image, but not run the container.

@svrnm @kpratyus @DebajitDas - I saw your blog article and was wondering if you were able to achieve the same goal with the nginx:alpine image?

docker file

FROM nginx:alpine
#FROM nginx:1.18
RUN apk update ; apk add unzip ca-certificates
#RUN apt-get update ; apt-get install unzip
ADD https://github.com/open-telemetry/opentelemetry-cpp-contrib/releases/download/webserver%2Fv1.0.3/opentelemetry-webserver-sdk-x64-linux.tgz /opt
RUN cd /opt ; unzip opentelemetry-webserver-sdk-x64-linux.tgz; tar xvfz opentelemetry-webserver-sdk-x64-linux.tgz
RUN cd /opt/opentelemetry-webserver-sdk; ./install.sh
ENV LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/opt/opentelemetry-webserver-sdk/sdk_lib/lib
RUN echo "load_module /opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/ngx_http_opentelemetry_module.so;\n$(cat /etc/nginx/nginx.conf)" > /etc/nginx/nginx.conf
COPY opentelemetry_module.conf /etc/nginx/conf.d

build container

docker build -t nginx-alpine-otel --platform linux/amd64 .

run container

docker run --platform linux/amd64 --rm -p 8080:80 nginx-alpine-otel

error message

[emerg] 1#1: dlopen() "/opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/ngx_http_opentelemetry_module.so" failed (Error loading shared library /opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/ngx_http_opentelemetry_module.so: No such file or directory) in /etc/nginx/nginx.conf:1
nginx: [emerg] dlopen() "/opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/ngx_http_opentelemetry_module.so" failed (Error loading shared library /opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/ngx_http_opentelemetry_module.so: No such file or directory) in /etc/nginx/nginx.conf:1

dmiyamasu avatar Jul 21 '23 14:07 dmiyamasu

The released modules are for glibc, so you need to build the modules from scratch as outlinded here. Note that also these instructions are for ubuntu & centos which are glibc based, so you need to take care of that part as well.

If you are curious, there is an alpine package for OpenTelemetry C++:

https://pkgs.alpinelinux.org/packages?name=opentelemetry-cpp-*

You may be able to use that one.

Also the demo makes use of Alpine:

https://github.com/open-telemetry/opentelemetry-demo/blob/main/src/currencyservice/Dockerfile

If you are successful, don't hesitate to share & contribute your results

svrnm avatar Jul 21 '23 15:07 svrnm

For what it's worth, Datadog's nginx module (based originally on nginx-opentracing) supports Alpine, Amazon Linux, and Debian on an image-by-image basis.

The only way I could think to do it is to have separate tooling for each package manager (apt, yum, apk). The integration tests also have to branch on the package manager.

Even with that, nginx-datadog supports certain docker images only. There are multiple issues about using the module elsewhere, and the failures are always due to some difference in:

  • nginx's ./configure flags (i.e. the module signature)
  • the version of nginx (has to match exactly, down to the patch version)
  • the version and flavor of libc (glibc, musl, too old...)
  • whether the nginx source has been patched relative to a source release (OpenResty does this)
  • the directory where modules are installed (a distro package maintainer might choose a different directory than that reported by nginx -V)
  • supported architecture (people are asking for ARM64 now)
  • compatibility with Darwin (for engineers and support representatives using Macbooks)

I'll be watching this repository to see if I can learn from any decisions you make with regard to those concerns. :)

dgoffredo avatar Jul 21 '23 18:07 dgoffredo

After much trying and failing, we did not make progress on this solution. Instead, we had to compromise and use the Debian based Docker nginx image. See my colleague's repo (@andrew-lozoya) here for a solution using Debian, NGINX open source, AWS OTEL collector and New Relic nri-ecs

Trying to compile the opentelemetry-cpp-contrib/webserver module for NGINX using the Alpine apk package manager ended up being a deep rabbit hole of compilation issues.

If this changes, will report back in the future.

dmiyamasu avatar Aug 15 '23 20:08 dmiyamasu

nginx: [emerg] dlopen() "/opt/opentelemetry-webserver-sdk/WebServerModule/Nginx/1.23.1/ngx_http_opentelemetry_module.so" failed (Error loading shared library libcrypt.so.1: No such file or directory (needed by /opt/opentelemetry-webserver-sdk/sdk_lib/lib/libopentelemetry_webserver_sdk.so)) in /etc/nginx/nginx.conf:1

I get this error when use the module in alpine images, go great if can provide support for alpine

kalingaru48 avatar Mar 05 '24 07:03 kalingaru48

Able to see traces with the apk from https://nginx.org/packages/mainline/alpine/v3.18/main/x86_64/nginx-module-otel-1.25.3.0.1.0-r1.apk

More info here: https://docs.nginx.com/nginx/admin-guide/dynamic-modules/opentelemetry/ https://github.com/nginxinc/nginx-otel?tab=readme-ov-file

Dockerfile:

RUN wget "https://nginx.org/packages/mainline/alpine/v3.18/main/x86_64/nginx-module-otel-1.25.3.0.1.0-r1.apk" RUN apk add --allow-untrusted nginx-module-otel-1.25.3.0.1.0-r1.apk

kiranvaddadi avatar Mar 05 '24 08:03 kiranvaddadi

@kiranvaddadi it worked! thanks

kalingaru48 avatar Mar 06 '24 06:03 kalingaru48

Dropping this here if someone else wants to use the solution provided by @kiranvaddadi This should in theory work even if the image is updated.

Dockerfile

FROM nginx:stable-alpine

RUN apk add --no-cache libstdc++

ARG ARCH="aarch64"

RUN ALPINE_VERSION=$(cat /etc/alpine-release | cut -d. -f1-2 | sed 's/^/v/') && \
    NGINX_VERSION=$(nginx -v 2>&1 | cut -d/ -f2) && \
    MODULE_URL_BASE=https://nginx.org/packages/alpine/$ALPINE_VERSION/main/${ARCH}/ && \
    wget -qO- $MODULE_URL_BASE | \
    grep -o 'nginx-module-otel-[0-9\.]*-r[0-9]*\.apk' | \
    sort -Vr | \
    head -n 1 | \
    xargs -I {} wget ${MODULE_URL_BASE}{} && \
    tar -xzf ./nginx-module-otel-*.apk && \
    rm nginx-module-otel-*.apk

nginx.conf

load_module modules/ngx_otel_module.so;

http {
    ...

    otel_exporter {
        endpoint otel_collector:4317;
    }
    otel_trace on;
    otel_service_name "nginx-proxy";

    server {
        ...

        location / {
            ...

            otel_trace_context propagate;
            otel_span_name "$request_method $request_uri";
        }
    }
}

chdanielmueller avatar Jul 18 '24 11:07 chdanielmueller