prom_ex icon indicating copy to clipboard operation
prom_ex copied to clipboard

Empty grafana dashboards

Open AbhishekBose opened this issue 2 years ago • 18 comments

I have setup prom_ex for my phoenix elixir application as defined in the tutorial. Trying to test out the prom_ex built in plugins:

  • Plugins.Application,
  • Plugins.Beam,
  • {Plugins.Phoenix, router: QuizRoomsWebWeb.Router, endpoint: QuizRoomsWebWeb.Endpoint}

The mix prom_ex.dashboard.publish also executes successfully. On running the application, I am expecting something to show up on the grafana dashboard panels, but it all seems to be empty. My prometheus server is running on localhost:9090 and grafana is running on localhost:3000. I have added the datasource to grafana successfully. Saving and testing works, but the output seems empty. Am I supposed to perform some extra steps here to populate grafana with the metrics. Please help

Screenshot 2021-10-12 at 11 07 09 PM

AbhishekBose avatar Oct 12 '21 17:10 AbhishekBose

Does the datasource name configured in the PromEx module align with the name of the datasource in Grafana?

akoutmos avatar Oct 12 '21 17:10 akoutmos

yep. configured the name as prometheus both on grafana and the promEx module.

Also, I cannot see the app in my prometheus targets page. Currently running grafana and prometheus directly on my local. Not using a docker container of any kind Screenshot 2021-10-13 at 12 06 51 AM

AbhishekBose avatar Oct 12 '21 18:10 AbhishekBose

Based on that screenshot it doesn't look like Prometheus has been configured to actually scrape your Elixir app. It looks like it is only scraping itself. Have you updated your Prometheus config file to hit your Elixir app? Something like: https://github.com/akoutmos/prom_ex/blob/master/example_applications/shared_docker/prometheus/config.yml#L15-L17

akoutmos avatar Oct 12 '21 18:10 akoutmos

I had totally missed that part out. Added prometheus config file. I can see the app on my prometheus dashboard now Screenshot 2021-10-13 at 12 54 20 AM

Getting data on the application dashboard as shown below. Screenshot 2021-10-13 at 12 55 09 AM

and also on the beam dashboard Screenshot 2021-10-13 at 12 58 23 AM

But nothing on the phoenix dashboard

Screenshot 2021-10-13 at 12 55 51 AM

Getting this error on the terminal

Screenshot 2021-10-13 at 12 57 46 AM

AbhishekBose avatar Oct 12 '21 19:10 AbhishekBose

I notice that the Phoenix Endpoint dropdown in the dashboard is set to None. I am guessing that in your PromEx config, perhaps the Endpoint module is incorrect? Also, what version of Phoenix are you running?

akoutmos avatar Oct 12 '21 19:10 akoutmos

this is how the configuration looks right now

endpoint.ex

Screenshot 2021-10-13 at 1 16 39 AM

prom_ex.ex

Screenshot 2021-10-13 at 1 16 53 AM

phoenix version = 1.6.2

AbhishekBose avatar Oct 12 '21 19:10 AbhishekBose

That is odd that it is dropping the telemetry handler given that your set up looks good to me. Is this a public repo that I can look at and perhaps run?

akoutmos avatar Oct 12 '21 20:10 akoutmos

This is a private repo unfortunately. I will create a dummy phoenix umbrella project similar to this one and share the repo for you to check

AbhishekBose avatar Oct 13 '21 18:10 AbhishekBose

Thanks :+1:

akoutmos avatar Oct 13 '21 18:10 akoutmos

@akoutmos This is the link to the dummy prom_ex test project --> https://github.com/AbhishekBose/prom_ex_test_project

Ran it on my local and getting similar errors for phoenix and liveview dashboards. Screenshots below

Screenshot 2021-10-15 at 1 01 33 PM Screenshot 2021-10-15 at 1 01 26 PM Screenshot 2021-10-15 at 1 00 53 PM

Please have a look

AbhishekBose avatar Oct 15 '21 07:10 AbhishekBose

@akoutmos Can you kindly have a look at this?

AbhishekBose avatar Oct 18 '21 16:10 AbhishekBose

Sorry about that. I totally missed your last message. I'll have a look at your test application in a couple days as I am a bit swamped at the moment. Thanks for putting that together! That will really help :)

akoutmos avatar Oct 18 '21 17:10 akoutmos

No problem. Thanks Looking forward!

AbhishekBose avatar Oct 18 '21 18:10 AbhishekBose

Hey @akoutmos any updates on this?

AbhishekBose avatar Oct 28 '21 16:10 AbhishekBose

This one is turning out to be quite a bit of work unfortunately :/. Still working on getting true umbrella support working in PromEx as this issue turns out to be the same issue as #31. That said, I probably need a couple more weeks till I land on an implementation I like.

akoutmos avatar Oct 29 '21 18:10 akoutmos

Cool. No problem. Will be following the thread.

AbhishekBose avatar Nov 03 '21 17:11 AbhishekBose

Getting this too, except not in an umbrella. All dashboards work but Phoenix

frozencap avatar Oct 29 '22 22:10 frozencap

@shawarmaz I was having the same problem until I enabled the Phoenix plugin in my lib/my_app/prom_ex.ex module by uncommenting the relevant lines in the plugins section and the dashboards section.

After that, I restarted the server and it was smooth sailing!

arcanemachine avatar May 23 '23 12:05 arcanemachine