cephmetrics icon indicating copy to clipboard operation
cephmetrics copied to clipboard

Published 20 hours ago • verified publisher icon ceph

templates parse error at character 6

Open stef97 opened this issue 7 years ago • 3 comments
trafficstars

using default ansible deployment (containers and prometheus) dashboards are showing but no data

After updating dashboard using dashUpdate.py and reloading grafana, the following error shows

Templating init failed parse error at char 6: could not parse remaining input ",osd02,osd04".

The only error I noticed ( docks logs grafan-server) is msg="can't read dashboard provisioning files from directory" logger=provisioning.dashboard path=/etc/grafana/provisioning/dashboards

Thanks

Here are excerpts from dashboard.yml osd_servers:

  • osd01
  • osd02
  • osd04

rgw_servers:

  • osd01

domain: tor.medavail.net

_credentials: user: admin password: admin

_grafana_port: 3000 _grafana_host: mon01.tor.medavail.net

stef97 avatar Aug 23 '18 19:08 stef97

I've assigned to Zack since he's the ansible guy :) In the meantime, can you confirm that the prometheus port on the mgr's is accessible. port 9283 and 9100 on the mgrs and 9100 on all other nodes should be accessible for the prometheus instance to scrape the data.

Also confirm that the prometheus.yml file actually has the scrape definitions for your nodes in place.

pcuzner avatar Aug 24 '18 03:08 pcuzner

running the playbook again fixed the template error

I can see cluster specific data ( pools, capacity, pg ...etc) but nothig host/OSD specific ( disks, network ..etc)

Example: ceph cluster dashbord is fully populated ceph backend storage has no data except "osd host capacity summary"

selinux and firewalld are disabled on all hosts

Port 9100 is open and available on all nodes netstat -lntp | grep 9100 tcp 0 0 0.0.0.0:9100 0.0.0.0:* LISTEN 13263/node_exporter

Port 9283 and 9100 are open on MGR server netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 0.0.0.0:9090 0.0.0.0:* LISTEN 1446/docker-proxy-c tcp 0 0 0.0.0.0:9283 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 0.0.0.0:8003 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 10.10.30.191:6789 0.0.0.0:* LISTEN 832/ceph-mon tcp 0 0 127.0.0.1:199 0.0.0.0:* LISTEN 825/snmpd tcp 0 0 0.0.0.0:9100 0.0.0.0:* LISTEN 556/node_exporter tcp 0 0 10.10.30.191:6800 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 823/sshd tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 4852/docker-proxy-c tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1196/master

[root@mon01 ~]# cat /var/lib/cephmetrics/prometheus.yml global: scrape_interval: 15s evaluation_interval: 15s

scrape_configs:

  • job_name: 'prometheus' static_configs:
    • targets: ['localhost:9090']
  • job_name: 'ceph' honor_labels: true static_configs:
    • targets: ['mon01.tor.medavail.net:9283'] labels: instance: 'ceph_cluster'
  • job_name: 'node' static_configs:
    • targets: ['osd01.tor.medavail.net:9100'] labels: instance: "osd01.tor.medavail.net"
    • targets: ['osd02.tor.medavail.net:9100'] labels: instance: "osd02"
    • targets: ['osd04.tor.medavail.net:9100'] labels: instance: "osd04.tor.medavail.net"
  • job_name: 'cephmetrics' static_configs:
    • targets: ['mon01.tor.medavail.net:9100'] labels: instance: "mon01"

On Thu, 23 Aug 2018 at 23:24, pcuzner [email protected] wrote:

I've assigned to Zack since he's the ansible guy :) In the meantime, can you confirm that the prometheus port on the mgr's is accessible. port 9283 and 9100 on the mgrs and 9100 on all other nodes should be accessible for the prometheus instance to scrape the data.

Also confirm that the prometheus.yml file actually has the scrape definitions for your nodes in place.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ceph/cephmetrics/issues/221#issuecomment-415639821, or mute the thread https://github.com/notifications/unsubscribe-auth/AJn1v5Q6ym3v8FgE7-OuMRbJIwoFySp4ks5uT3IJgaJpZM4WKHEv .

stef97 avatar Aug 24 '18 12:08 stef97

Just to clarify

running playbook again fixed the issue HOWEVER, running "python dashUpdater.py" broke it again ( without making any changes to dashboard.yml

In summary, dashUpdater.py appears to do break the templates

Hers i the output in debug mode

[cephuser@ceph cephmetrics]$ python dashUpdater.py -d Connection to Grafana is ok Templates to update: {'domain': 'tor.medavail.net', 'osd_servers': ['osd01', 'osd02', 'osd04'], 'rgw_servers': ['osd01']}

Processing dashboard alert-status

  • fetch of alert-status from Grafana successful
  • existing alert dashboard found, update bypassed

Processing dashboard ceph-at-a-glance

  • fetch of ceph-at-a-glance from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': u'', u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Host', u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': u'', u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Host', u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True}
  • 1 templating variables updated : osd_servers
  • dashboard update successful
  • checking 'ceph-at-a-glance' is starred
  • fetch of ceph-at-a-glance from Grafana successful
  • setting home dashboard complete

Processing dashboard ceph-backend-storage

  • fetch of ceph-backend-storage from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'/instance="([^"])"/', u'sort': 3, u'multi': True, u'hide': 0, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Hostname', u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'regex': u'/instance="([^"])"/', u'sort': 3, u'multi': True, u'hide': 0, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Hostname', u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True}
  • 1 templating variables updated : osd_servers
  • dashboard update successful

Processing dashboard ceph-cluster

  • fetch of ceph-cluster from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved
  • 0 templating variables updated :
  • dashboard update successful

Processing dashboard ceph-health

  • fetch of ceph-health from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'/instance="([^"])"/', u'sort': 3, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': False} after {u'regex': u'/instance="([^"])"/', u'sort': 3, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': False}
  • 1 templating variables updated : osd_servers
  • dashboard update successful

Processing dashboard ceph-osd-information

  • fetch of ceph-osd-information from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved
  • 0 templating variables updated :
  • dashboard update successful

Processing dashboard ceph-pools

  • fetch of ceph-pools from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved
  • 0 templating variables updated :
  • dashboard update successful

Processing dashboard ceph-rgw-workload

  • fetch of ceph-rgw-workload from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'rgw_servers' before {u'regex': u'', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'rgw_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {u'text': u'osd01', u'value': u'osd01'}, u'datasource': None, u'type': u'query', u'query': u'osd01', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'osd01', u'selected': True, u'value': u'osd01'}], u'includeAll': True} after {u'regex': u'', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'rgw_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {'text': 'osd01', 'value': 'osd01'}, u'datasource': None, u'type': u'query', u'query': 'osd01', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'osd01', 'selected': True, 'value': 'osd01'}], u'includeAll': True}
  • 1 templating variables updated : rgw_servers
  • dashboard update successful

Processing dashboard disk-busy-by-server

  • fetch of disk-busy-by-server from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True}
  • 1 templating variables updated : osd_servers
  • dashboard update successful

Processing dashboard iops-by-server

  • fetch of iops-by-server from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': u'', u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Host', u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': u'', u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Host', u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True}
  • 1 templating variables updated : osd_servers
  • dashboard update successful

Processing dashboard iscsi-overview

  • fetch of iscsi-overview from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved
  • 0 templating variables updated :
  • dashboard update successful

Processing dashboard latency-by-server

  • fetch of latency-by-server from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'regex': u'([^.]).', u'sort': 1, u'multi': False, u'hide': 2, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': None, u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True}
  • 1 templating variables updated : osd_servers
  • dashboard update successful

Processing dashboard network-usage-by-node

  • fetch of network-usage-by-node from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'multi': True, u'hide': 0, u'name': u'osd_servers', u'allValue': None, u'label': u'OSD Hostname', u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'query': u'osd01,osd02,osd04', u'type': u'custom', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'multi': True, u'hide': 0, u'name': u'osd_servers', u'allValue': None, u'label': u'OSD Hostname', u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'query': 'osd01,osd02,osd04', u'type': u'custom', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True} processing variable 'domain' before {u'multi': False, u'hide': 2, u'name': u'domain', u'allValue': None, u'label': None, u'current': {u'text': u'tor.medavail.net', u'value': u'tor.medavail.net'}, u'query': u'tor.medavail.net', u'type': u'custom', u'options': [{u'text': u'tor.medavail.net', u'selected': True, u'value': u' tor.medavail.net'}], u'includeAll': False} after {u'multi': False, u'hide': 2, u'name': u'domain', u'allValue': None, u'label': None, u'current': {'text': 'tor.medavail.net', 'value': ' tor.medavail.net'}, u'query': 'tor.medavail.net', u'type': u'custom', u'options': [{'text': 'tor.medavail.net', 'selected': True, 'value': ' tor.medavail.net'}], u'includeAll': False} processing variable 'rgw_servers' before {u'multi': False, u'hide': 0, u'name': u'rgw_servers', u'allValue': None, u'label': u'RadosGW Host', u'current': {u'text': u'osd01', u'value': u'osd01'}, u'query': u'osd01', u'type': u'custom', u'options': [{u'text': u'osd01', u'selected': True, u'value': u'osd01'}], u'includeAll': True} after {u'multi': False, u'hide': 0, u'name': u'rgw_servers', u'allValue': None, u'label': u'RadosGW Host', u'current': {'text': 'osd01', 'value': 'osd01'}, u'query': 'osd01', u'type': u'custom', u'options': [{'text': 'osd01', 'selected': True, 'value': 'osd01'}], u'includeAll': True}
  • 3 templating variables updated : osd_servers,domain,rgw_servers
  • dashboard update successful

Processing dashboard osd-node-detail

  • fetch of osd-node-detail from Grafana successful
  • existing dashboard will be updated
  • dashboard retrieved processing variable 'osd_servers' before {u'regex': u'([^.]).', u'sort': 3, u'multi': False, u'hide': 0, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Host Name', u'current': {u'text': u'All', u'selected': True, u'value': u'$__all'}, u'datasource': None, u'type': u'query', u'query': u'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{u'text': u'All', u'selected': True, u'value': u'$__all'}, {u'text': u'osd01', u'selected': False, u'value': u'osd01'}, {u'text': u'osd02', u'selected': False, u'value': u'osd02'}, {u'text': u'osd04', u'selected': False, u'value': u'osd04'}], u'includeAll': True} after {u'regex': u'([^.]).', u'sort': 3, u'multi': False, u'hide': 0, u'name': u'osd_servers', u'tags': [], u'allValue': None, u'tagValuesQuery': u'', u'refresh': 1, u'label': u'OSD Host Name', u'current': {'text': 'All', 'selected': True, 'value': '$__all'}, u'datasource': None, u'type': u'query', u'query': 'osd01,osd02,osd04', u'useTags': False, u'tagsQuery': u'', u'options': [{'text': 'All', 'selected': True, 'value': '$__all'}, {'text': 'osd01', 'selected': False, 'value': 'osd01'}, {'text': 'osd02', 'selected': False, 'value': 'osd02'}, {'text': 'osd04', 'selected': False, 'value': 'osd04'}], u'includeAll': True}
  • 1 templating variables updated : osd_servers
  • dashboard update successful

Many thanks Steven

On Fri, 24 Aug 2018 at 08:02, Steven Vacaroaia [email protected] wrote:

running the playbook again fixed the template error

I can see cluster specific data ( pools, capacity, pg ...etc) but nothig host/OSD specific ( disks, network ..etc)

Example: ceph cluster dashbord is fully populated ceph backend storage has no data except "osd host capacity summary"

selinux and firewalld are disabled on all hosts

Port 9100 is open and available on all nodes netstat -lntp | grep 9100 tcp 0 0 0.0.0.0:9100 0.0.0.0:* LISTEN 13263/node_exporter

Port 9283 and 9100 are open on MGR server netstat -lntp Active Internet connections (only servers) Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name tcp 0 0 0.0.0.0:8443 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 0.0.0.0:9090 0.0.0.0:* LISTEN 1446/docker-proxy-c tcp 0 0 0.0.0.0:9283 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 0.0.0.0:8003 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 10.10.30.191:6789 0.0.0.0:* LISTEN 832/ceph-mon tcp 0 0 127.0.0.1:199 0.0.0.0:* LISTEN 825/snmpd tcp 0 0 0.0.0.0:9100 0.0.0.0:* LISTEN 556/node_exporter tcp 0 0 10.10.30.191:6800 0.0.0.0:* LISTEN 830/ceph-mgr tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 823/sshd tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 4852/docker-proxy-c tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 1196/master

[root@mon01 ~]# cat /var/lib/cephmetrics/prometheus.yml global: scrape_interval: 15s evaluation_interval: 15s

scrape_configs:

  • job_name: 'prometheus' static_configs:
    • targets: ['localhost:9090']
  • job_name: 'ceph' honor_labels: true static_configs:
    • targets: ['mon01.tor.medavail.net:9283'] labels: instance: 'ceph_cluster'
  • job_name: 'node' static_configs:
    • targets: ['osd01.tor.medavail.net:9100'] labels: instance: "osd01.tor.medavail.net"
    • targets: ['osd02.tor.medavail.net:9100'] labels: instance: "osd02"
    • targets: ['osd04.tor.medavail.net:9100'] labels: instance: "osd04.tor.medavail.net"
  • job_name: 'cephmetrics' static_configs:
    • targets: ['mon01.tor.medavail.net:9100'] labels: instance: "mon01"

On Thu, 23 Aug 2018 at 23:24, pcuzner [email protected] wrote:

I've assigned to Zack since he's the ansible guy :) In the meantime, can you confirm that the prometheus port on the mgr's is accessible. port 9283 and 9100 on the mgrs and 9100 on all other nodes should be accessible for the prometheus instance to scrape the data.

Also confirm that the prometheus.yml file actually has the scrape definitions for your nodes in place.

— You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub https://github.com/ceph/cephmetrics/issues/221#issuecomment-415639821, or mute the thread https://github.com/notifications/unsubscribe-auth/AJn1v5Q6ym3v8FgE7-OuMRbJIwoFySp4ks5uT3IJgaJpZM4WKHEv .

stef97 avatar Aug 24 '18 12:08 stef97