Unable to load Grafana config and graphs page after reverse proxy

[NC-Yungd]

Expected Behavior

Users should be able to load the Grafana configuration and graphs page through the Icingaweb2 web page (Configuration > Modules > grafana > configuration or graphs)

Current Behavior

After setting up reverse proxy for Grafana, whenever I try to view the Grafana configuration or graphs page, I get a hanging white screen with the Grafana logo and "Loading Grafana"

Below are the 404 errors:

10.5.50.167 - - [11/Dec/2019:18:32:03 +0000] "GET /config/modules HTTP/1.1" 200 3978 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /css/icinga.min.css HTTP/1.1" 304 216 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /js/icinga.min.js HTTP/1.1" 304 216 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /img/icinga-logo.svg HTTP/1.1" 304 181 "http://fakedomain.com/css/icinga.min.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /img/icons/search_white.png HTTP/1.1" 304 180 "http://fakedomain.com/css/icinga.min.css" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /grafana/config HTTP/1.1" 200 7633 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /public/build/runtime.189a5a829d25dac84fc0.js HTTP/1.1" 404 1110 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /public/build/grafana.dark.189a5a829d25dac84fc0.css HTTP/1.1" 404 13317 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /public/build/angular~app.189a5a829d25dac84fc0.js HTTP/1.1" 404 1117 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /public/build/app.189a5a829d25dac84fc0.js HTTP/1.1" 404 1101 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /public/build/moment~app.189a5a829d25dac84fc0.js HTTP/1.1" 404 1115 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"
10.5.50.167 - - [11/Dec/2019:18:32:04 +0000] "GET /public/build/vendors~app.189a5a829d25dac84fc0.js HTTP/1.1" 404 1117 "http://fakedomain.com/config/modules" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:70.0) Gecko/20100101 Firefox/70.0"

Possible Solution

Steps to Reproduce (for bugs)

1. Login to your Icingaweb2 web page
2. Go to Configuration > Modules > grafana
3. Click on either the configurations or graphs tab
4. You will see a jumping Grafana logo with the words "Loading Grafana"

Changes that I made on the server:

/etc/grafana/grafana.ini

[server]
domain = fakedomain.com
root_url = %(protocol)s://%(domain)s/grafana

/etc/apache2/sites-available/apache.conf

ProxyPass /grafana http://fakedomain.com:3000
ProxyPassReverse /grafana http://fakedomain.com:3000

/etc/icingaweb2/modules/grafana/config.ini

[grafana]
host = "fakedomain.com/grafana"

Context

After setting up reverse proxy, I am unable to update Grafana configurations through the Icingaweb2 modules page. However, when I revert the changes for reverse proxy, I am able to access the Grafana configurations and graphs page.

This doesn't seem like a huge issues since I can edit the config.ini and graphs.ini through the server, but I would like the option to be able to do it from the web browser.

Your Environment

• Icinga Web 2 Grafana version (System - About): 1.1.7
• Icinga Web 2 version and modules (System - About): 2.7.3
• Version used (icinga2 --version): r2.11.2-1
• Operating System and version: Ubuntu 18.04
• Enabled features (icinga2 feature list): Disabled features: compatlog debuglog elasticsearch gelf graphite livestatus opentsdb perfdata statusdata syslog Enabled features: api checker command ido-mysql influxdb mainlog notification

[Mikesch-mp]

This is a problem of your reverse proxy settings. The modules does not contact or load anything from grafana on graph configs or module configuration page. Best thing you can do to debug your problem is opening the dev tools from the browser .

[sebo1312]

My working Setup:

/etc/grafana/grafana.ini

[server] ;domain = root_url = %(protocol)s://%(domain)s/grafana

/etc/icingaweb2/modules/grafana/config.ini - see also #231

publichost = "../../../grafana"

Apache:

RewriteRule ^/grafana(.*) http://127.0.0.1:3000$1 [P]

[NC-Yungd]

@sebo1312 After applying those settings, I was able to load the Grafana module from inside Icinga Web. However, it broke Grafana UI when I try to visit fakedomain.com/grafana

[Mikesch-mp]

Btw the module uses the path /grafana for itself. Normaly you have /icingaweb2/grafana , but if you remove the icingaweb2 part you have 2 identical pathes.

[Mikesch-mp]

@grant-veepshosting your url is complete wrong. Grafana uses either a port (default is 3000) or you put it behind a proxy (apache/nginx) and use url only on http or https.

[grant-veepshosting]

@grant-veepshosting your url is complete wrong. Grafana uses either a port (default is 3000) or you put it behind a proxy (apache/nginx) and use url only on http or https.

Removed as incorrect, apologies..