carbonapi icon indicating copy to clipboard operation
carbonapi copied to clipboard

Improve balancer method "rr" logic to be able to retry empty replies

Open proffust opened this issue 4 years ago • 5 comments

При недоступности одной server group carbonapi отвечает на запрос значений тегов, например /tags/autoComplete/values?limit=10000&tag=cluster кодом 500, при этом получает данные от бекендов Логи https://pastebin.com/5FTPvwN9

proffust avatar Jul 16 '20 15:07 proffust

What version of carbonapi do you use?

In current master I've made a commit yesterday that I think should fix this issue: https://github.com/go-graphite/carbonapi/commit/2ef98182f2e3e58a1601fcb0d7a34ba20f4d9adb

Civil avatar Jul 16 '20 16:07 Civil

Ok, from the logs I can see that you are running 0.14.0, could you please verify that current master works for your use-case?

Civil avatar Jul 16 '20 16:07 Civil

500 should no longer happen in current master.

However if you use lbMethod: "rr" and one of the servers returns nothing, retry won't happen as it's not possible to determine if that's by design or that's an issue with one of the backends.

If you want, we can discuss how retry behavior should change in that case, otherwise I would consider this issue as "fixed".

Civil avatar Jul 16 '20 20:07 Civil

I suggest adding a parameter to the config, whether to ignore errors from this group server or not

proffust avatar Jul 17 '20 08:07 proffust

@Civil, this is also a setting that we'd be interested in. We use rr and our backends will always return something for a valid response. If a backend host goes down it would be good if it assumed that an empty response was a bad response and that it would try the next host.

jonasbleyl avatar Sep 14 '20 17:09 jonasbleyl