nginxinc
Not able to access a prometheus endpoint
I am deploying a triton inference server on the docker swarm and using Nginx for load-balancing. Since triton inference server has three endpoints:
port 8000: for HTTP requests
port 8001: for grpc requests
port 8002: Prometheus metrics server
I am creating an overlay network and starting a service of triton and it converges. Then, my Nginx service for grpc requests is also working. I was not able to route to 8002 to get Prometheus metrics, so I tries using two load balancers one for each endpoint.
The nginx.conf file for the 8002 endpoint Nginx looks like this:
upstream metrics {
server triton-svc:8002;
}
server {
listen 8092;
location / {
proxy_pass http://metrics;
}
}
where triton-svc is the triton service. I am creating the service using this command:
$ sudo docker service create --name nginx-metrics -p 8092:8092 --network appnetwork nginxmetrics:v6
where nginxmetrics:v6 is the nginx image with the config above.
The service converges and logs are fine. But I am not able to access the logs at http:localhost:8092/metrics
I have two questions:
- Do I need to use two nginx LoadBalancers in this case or one will work? If yes, how will the config look like?
- If I need two LBs, one for each endpoint. How to configure my nginx to get the metrics at 8002?
Hi @sourabh-burnwal!
One NGINX load balancer should work perfectly fine in this scenario. Check your firewalls or network setup if you're unable to hit the port 8092, it does not seem to be related to NGINX too.
