otel-with-apache-pulsar
otel-with-apache-pulsar copied to clipboard
riferrei
Example of application that produces and consumes events to/from Apache Pulsar. Traces from the transactions are captured using OpenTelemetry and sent to Elastic Observability.
OpenTelemetry with Apache Pulsar
This repository contains an example of application that offers users a way to estimate the cost of certain car brands. When a brand is provided the application queries the estimative in a Redis database. Along with the operation to estimate, the application also produces and consumes events to and from Apache Pulsar. This whole transaction is captured using OpenTelemetry and sent to Elastic Observability.
By using Elastic APM, folks from Ops gain instant insight into transactions, granting them a way to pragmatically observe when the transaction becomes unavailable or when it contains bottlenecks. More importantly, it allows them to know exactly where the transaction is the problem. Here is an example of a transaction timeline from the application:
As you can see in the timeline above, as part of the estimate operation, a step feeds the analytics platform currently implemented using Pulsar. The child operations estimates send and estimates receive represent the usage of producers and consumers from Pulsar in the code — that write and read messages in a topic called estimates.
This is the part that this repository aims to address. Because OpenTelemetry currently has no official support for Pulsar, the usage of Pulsar APIs in this application would create "black holes" in the transaction, which then compromises the Ops team ability to diagnose, for example — when the producer or the consumer from Pulsar could be affecting the transaction's availability or performance.
To address this problem, custom instrumentation for Pulsar was created.
Pre-requisites
The following software must be correctly installed in the machine to run this example.
| Docker | https://docs.docker.com/get-docker/ |
Getting started
Open a new terminal and execute the following steps:
1. Clone the repository
git clone https://github.com/riferrei/otel-with-apache-pulsar.git
2. Start the microservice
docker compose up -d
3. Execute HTTP requests
curl -X GET http://localhost:8888/estimate?brand=ferrari
Accessing Elastic Observability
After executing the services you can reach the Elastic Observability application in the following URL:
http://localhost:5601/app/apm/services
Use the following credentials:
User: admin
Pass: changeme
License
This project is licensed under the Apache 2.0 License.
riferrei