Exception on /render [POST]

[ihard]

I use Grafana 3.1 + nginx + uwsgi + graphite-api 1.1.3 + go-carbon and some charts do not display data in log uwsgi: {"pattern": "..metric.count", "start": 1468443600, "finder": "whisper", "end": 1468504998, "event": "find_nodes"} {"path": "/var/lib/graphite/whisper/test/host1/metric/count.wsp", "reader": "whisper", "start": 1468443600, "metric_path": "test.host1.metric.count", "end": 1468504998, "event": "fetch"} {"event": "Exception on /render [POST]", "exception": "Traceback (most recent call last):\n File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1988, in wsgi_app\n response = self.full_dispatch_request()\n File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1641, in full_dispatch_request\n rv = self.handle_user_exception(e)\n File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1544, in handle_user_exception\n reraise(exc_type, exc_value, tb)\n File "/usr/local/lib/python3.5/dist-packages/flask/_compat.py", line 33, in reraise\n raise value\n File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1639, in full_dispatch_request\n rv = self.dispatch_request()\n File "/usr/local/lib/python3.5/dist-packages/flask/app.py", line 1625, in dispatch_request\n return self.view_functionsrule.endpoint\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/app.py", line 375, in render\n data_store = fetchData(context, paths)\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/render/datalib.py", line 167, in fetchData\n (node, node.fetch(startTime, endTime)) for node in single_nodes]\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/render/datalib.py", line 167, in \n (node, node.fetch(startTime, endTime)) for node in single_nodes]\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/node.py", line 28, in fetch\n return self.reader.fetch(startTime, endTime)\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/finders/whisper.py", line 121, in fetch\n cached_datapoints = self.carbonlink.query(self.real_metric_path)\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/carbonlink.py", line 199, in query\n results = self.send_request(request)\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/carbonlink.py", line 230, in send_request\n host = self.select_host(metric)\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/carbonlink.py", line 160, in select_host\n for node in self.hash_ring.get_nodes(key):\n File "/usr/local/lib/python3.5/dist-packages/graphite_api/carbonlink.py", line 108, in get_nodes\n index = bisect.bisect_left(self.ring, search_entry) % self.ring_len\nTypeError: unorderable types: tuple() < NoneType()"}

[krakatoa]

I have the same configuration and issue, with Grafana+nginx+graphite-api+go-carbon, still not sure if it's caused by the fact I'm using go-carbon, the error log is on graphite-api:

$ curl "http://192.168.1.104:8899/render?target=sumSeries(ch-*.online_count)"

Traceback (most recent call last): File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1988, in wsgi_app response = self.full_dispatch_request() File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1641, in full_dispatch_request rv = self.handle_user_exception(e) File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1544, in handle_user_exception reraise(exc_type, exc_value, tb) File "/usr/local/lib/python3.4/dist-packages/flask/_compat.py", line 33, in reraise raise value File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1639, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.4/dist-packages/flask/app.py", line 1625, in dispatch_request return self.view_functionsrule.endpoint File "/usr/local/lib/python3.4/dist-packages/graphite_api/app.py", line 375, in render data_store = fetchData(context, paths) File "/usr/local/lib/python3.4/dist-packages/graphite_api/render/datalib.py", line 167, in fetchData (node, node.fetch(startTime, endTime)) for node in single_nodes] File "/usr/local/lib/python3.4/dist-packages/graphite_api/render/datalib.py", line 167, in (node, node.fetch(startTime, endTime)) for node in single_nodes] File "/usr/local/lib/python3.4/dist-packages/graphite_api/node.py", line 28, in fetch return self.reader.fetch(startTime, endTime) File "/usr/local/lib/python3.4/dist-packages/graphite_api/finders/whisper.py", line 121, in fetch cached_datapoints = self.carbonlink.query(self.real_metric_path) File "/usr/local/lib/python3.4/dist-packages/graphite_api/carbonlink.py", line 199, in query results = self.send_request(request) File "/usr/local/lib/python3.4/dist-packages/graphite_api/carbonlink.py", line 230, in send_request host = self.select_host(metric) File "/usr/local/lib/python3.4/dist-packages/graphite_api/carbonlink.py", line 160, in select_host for node in self.hash_ring.get_nodes(key): File "/usr/local/lib/python3.4/dist-packages/graphite_api/carbonlink.py", line 108, in get_nodes index = bisect.bisect_left(self.ring, search_entry) % self.ring_len TypeError: unorderable types: tuple() < NoneType()

[iain-buclaw-sociomantic]

Maybe post the carbon config in graphite-api.yaml? It may be reproducible in the testsuite given the right environment conditions.

[krakatoa]

Mine is:

whisper:
  directories:
    - /var/lib/graphite/whisper
carbon:
  hosts:
    - 127.0.0.1:7002
allowed_origins:
  - 192.168.1.81:3000
finders:
  - graphite_api.finders.whisper.WhisperFinder
functions:
  - graphite_api.functions.SeriesFunctions
  - graphite_api.functions.PieFunctions

[kkcz]

Got the same error. line 108, in get_nodes index = bisect.bisect_left(self.ring, search_entry) % self.ring_len TypeError: unorderable types: tuple() < NoneType()

Looks like python2 x python3 incompatibility, so I changed line 108 from search_entry = position, None to search_entry = position,

and it works now.

python 3.5.3 graphite-api 1.1.3