open-telemetry
OTLP exporter v1.22.0 - Resulting deployment is not correct
Describe your environment
RH 8.10
Installed everything from source for OTLP sdk/exporters only
ci/install_thirdparty.sh --install-dir /home/gp/usr/local/otelLibs --tags-file install/cmake/third_party_latest --packages "abseil;ms-gsl;zlib;protobuf;grpc;ms-gsl;opentelemetry-proto"
For otel build
cmake -DCMAKE_INSTALL_PREFIX=/home/gp/usr/local/otel ..
-- gRPC_CPP_PLUGIN_EXECUTABLE=$<TARGET_FILE:gRPC::grpc_cpp_plugin> -- PROTOBUF_PROTOC_EXECUTABLE=$<TARGET_FILE:protobuf::protoc>
-- build settings
-- OpenTelemetry: 1.22.0 -- OpenTelemetry ABI: 2 -- CMAKE_SYSTEM_PROCESSOR: x86_64 -- CXX: GNU 8.5.0 -- CMAKE_BUILD_TYPE: Release -- CXXFLAGS: -- CMAKE_CXX_STANDARD: 17 -- CMAKE_TOOLCHAIN_FILE: -- BUILD_SHARED_LIBS:
-- opentelemetry-cpp build options
-- WITH_API_ONLY: OFF -- WITH_NO_DEPRECATED_CODE: ON -- WITH_ABI_VERSION_1: OFF -- WITH_ABI_VERSION_2: ON -- OTELCPP_VERSIONED_LIBS: OFF -- OTELCPP_MAINTAINER_MODE: OFF -- WITH_STL: CXX17 -- WITH_GSL: OFF -- WITH_NO_GETENV: OFF
-- opentelemetry-cpp cmake component options
-- WITH_OTLP_GRPC: ON -- WITH_OTLP_HTTP: OFF -- WITH_OTLP_FILE: OFF -- WITH_HTTP_CLIENT_CURL: OFF -- WITH_ZIPKIN: OFF -- WITH_PROMETHEUS: OFF -- WITH_ELASTICSEARCH: OFF -- WITH_OPENTRACING: OFF -- WITH_ETW: -- OPENTELEMETRY_BUILD_DLL:
-- feature preview options
-- WITH_ASYNC_EXPORT_PREVIEW: ON -- WITH_THREAD_INSTRUMENTATION_PREVIEW: ON -- WITH_METRICS_EXEMPLAR_PREVIEW: ON -- WITH_OTLP_GRPC_SSL_MTLS_PREVIEW: ON -- WITH_OTLP_GRPC_CREDENTIAL_PREVIEW: ON -- WITH_OTLP_RETRY_PREVIEW: ON
-- third-party options
-- WITH_NLOHMANN_JSON: OFF -- WITH_CURL_LOGGING: OFF -- WITH_OTLP_HTTP_COMPRESSION: OFF
-- examples and test options
-- WITH_BENCHMARK: OFF -- WITH_EXAMPLES: ON -- WITH_EXAMPLES_HTTP: OFF -- WITH_FUNC_TESTS: OFF -- BUILD_W3CTRACECONTEXT_TEST: OFF -- BUILD_TESTING: OFF
-- versions
-- CMake: 3.26.5 -- GTest: () -- benchmark: () -- Abseil: 20240722 -- opentelemetry-proto: 1.7.0 (fetch_repository) -- Protobuf: 30.2.0 (find_package - STATIC_LIBRARY) -- gRPC: 1.72.1 (find_package - STATIC_LIBRARY) -- ZLIB: 1.2.11 ()
Build & deployment succeeds.
What is the expected behavior?
"make install" will deploy all needed artefacts.
Setting the below in my CMakeList.txt to be sufficient
list(APPEND CMAKE_PREFIX_PATH "/home/gp/usr/local/otel/lib64/cmake/") find_package(opentelemetry-cpp CONFIG REQUIRED)
What is the actual behavior?
CMake fails, it tries to find libraries I thought are only needed for the SDK building
-- get_installed_components: component = api, installed = TRUE -- get_installed_components: component = sdk, installed = TRUE -- get_installed_components: component = ext_common, installed = TRUE -- get_installed_components: component = exporters_otlp_common, installed = TRUE -- get_installed_components: component = exporters_otlp_grpc, installed = TRUE -- get_installed_components: component = exporters_ostream, installed = TRUE -- get_installed_components: component = exporters_in_memory, installed = TRUE -- get_requested_components: Components requested: api;sdk;exporters_otlp_grpc -- find_required_dependencies: dependency = Microsoft.GSL, is_required = FALSE -- find_required_dependencies: dependency = ZLIB, is_required = FALSE -- find_required_dependencies: dependency = CURL, is_required = FALSE -- find_required_dependencies: dependency = nlohmann_json, is_required = FALSE -- find_required_dependencies: dependency = Protobuf, is_required = TRUE -- find_required_dependencies: calling find_dependency(Protobuf CONFIG )...
CMake Error at /usr/share/cmake/Modules/CMakeFindDependencyMacro.cmake:76 (find_package): Could not find a package configuration file provided by "Protobuf" with any of the following names:
ProtobufConfig.cmake
protobuf-config.cmake
Add the installation prefix of "Protobuf" to CMAKE_PREFIX_PATH or set "Protobuf_DIR" to a directory containing one of the above files. If "Protobuf" provides a separate development package or SDK, be sure it has been installed.
Have I done something wrong or my expectation, the OTEL public API doesnt pull any third party dependency, is wrong?
Hi @gparlamas, thanks for testing this and sharing the CMake logs.
From the command you shared the ci/install_thirdparty.sh script will install the dependencies (Protobuf, gRPC, ...) to /home/gp/usr/local/otelLibs and your CMake command will install opentelemetry-cpp to /home/gp/usr/local/otel. However, when you configure your project only the latter path is set to CMAKE_PREFIX_PATH.
Adding the install path of opentelemetry-cpp (/home/gp/usr/local/otel) and the install path of the dependencies (/home/gp/usr/local/otelLibs) to CMAKE_PREFIX_PATH should allow Protobuf to be found.
Hi @dbarker,
I am not using grpc / proto /absl in my system so would like to minimise 3rd party dependencies. I thought / was hoping the static build of otlp libs will contain everything that's needed / pull all deps and link everything into static lib or "make install" will pull everything required.
If I use the same installation path for both, ci/install_thirdparty.sh deploys lots of libraries I don't need so will have to manually start deleting stuff till the build breaks. Not ideal.
this is how it looks if path is the same
ls /home/gp/usr/local/include/ absl ares_dns.h ares_nameser.h ares_version.h grpc grpcpp opentelemetry upb utf8_validity.h zlib.h ares_build.h ares.h ares_rules.h google grpc++ gsl re2 utf8_range.h zconf.h
ls /home/gp/usr/local/lib/ cmake libgpr.a libgrpc_authorization_provider.a libgrpc++_reflection.a libssl.a libupb_message_lib.a libutf8_range_lib.a libz.so.1.3.1 libaddress_sorting.a libgrpc++.a libgrpc++_error_details.a libgrpc++_unsecure.a libupb_base_lib.a libupb_mini_descriptor_lib.a libz.a pkgconfig libcares.a libgrpc.a libgrpc_plugin_support.a libgrpc_unsecure.a libupb_json_lib.a libupb_textformat_lib.a libz.so libcrypto.a libgrpc++_alts.a libgrpcpp_channelz.a libre2.a libupb_mem_lib.a libupb_wire_lib.a libz.so.1
ls /home/gp/usr/local/lib64/
Not showing lib64, too many libs
From include dir above I only need opentelemetry folder for my app
@gparlamas Thanks for the context. From your original CMake logs WITH_OTLP_GRPC=ON which will build the otlp grpc exporters (requiring Abseil, Protobuf, zlib and gRPC). Protobuf will be a public transitive dependency from opentelemetry-cpp to your project, while gRPC is private to opentelemetry-cpp. CMake therefore needs to find both (and these have a transitive dependency on abseil and zlib).
When you call find_package(opentelemetry-cpp CONFIG) it will call find_dependency(Protobuf CONFIG) and find_dependency(gRPC CONFIG) implicitly because CMake needs to know how to find these dependencies (public or private) - this is a consequence of opentelemetry-cpp's CMake calling target_link_libraries on Protobuf and gRPC targets without restricting to just the BUILD_INTERFACE.
There is some room for cleanup in your commands. Since you are not setting WITH_GSL=ON you can remove ms-gsl from the install and allow opentelemetry-proto to be fetched from GitHub.
This should work for your setup:
ci/install_thirdparty.sh --install-dir /home/gp/usr/local/otelLibs --tags-file install/cmake/third_party_latest --packages "abseil;zlib;protobuf;grpc;"
Thanks @dbarker It's not ideal though for systems that are not using directly these 3, big, libraries (absl, proto, grpc). Otel is forcing us to deploy them.
I noticed as well by looking at build.make/link.txt files cmake generates for a target that uses otlp that the target depends on pretty much every library I manually build, ~107 in total! libabsl_* /libupb_* /libgrpc.a/libprotobuf.a/libgrpc++.a/libopentelemetry_* etc
@gparlamas
Can you clarify which exporter the application will use precisely ?
The build shows WITH_OTLP_GRPC=ON, is this intended ?
Hi @marcalff
I am using / exporting metrics only using otlp protocol over grpc. Using find_package(opentelemetry-cpp CONFIG REQUIRED COMPONENTS api exporters_otlp_grpc)
I am building a mini lib with target_link_libraries(otel-lib PUBLIC opentelemetry-cpp::api PRIVATE opentelemetry-cpp::otlp_grpc_metrics_exporter)
and apps link against it
target_link_libraries(myApp PUBLIC otel-lib)
I chose otlp-grpc for performance reasons / control, binary protocol / can pin exporter thread with opentelemetry::sdk::common::ThreadInstrumentation::OnStart()
Prometheus, which I have build too, its quite small library so none of the above worries apply. Unfortunately Its a text protocol and I don't understand its threading model - which thread runs the web server / can I control it?
@gparlamas
Thanks for the details.
If using the OTLP GRPC exporter, then yes, this will bring a lot of dependencies due to grpc.
I don't have performance numbers to compare, but note that the OTLP HTTP exporter can send binary as well, with protobuf encoding.
@marcalff Indeed, otlp-http doesn't rely on gRPC, deploying OpenTelemetry dependencies is significantly less noisy and more lightweight.
![github-actions[bot] avatar](https://avatars.githubusercontent.com/in/15368?v=4 "Published by a pub.dev verified publisher"){kind=small}
@gparlamas We tested OTLP HTTP and OTLP gRPC exporters a long time ago. Back then, OTLP HTTP did not reuse connections and consumed about 200% more CPU than OTLP gRPC. We have not re-benchmarked since OTLP HTTP added connection reuse, which is now the default.
prometheus-cpp’s push API uses curl’s synchronous API and launches new threads via std::async for exporting. This blocks only the I/O thread, but the number of threads is uncontrolled. The prometheus-cpp pull API (used in this repo) relies on civetweb, which defaults to 2 threads.
otel-cpp’s Prometheus exporter currently lacks options to configure civetweb’s thread settings for Prometheus. Contributions are welcome.
