consul_raft_leader does not reflect correct state of leader

[Abhimanyu-Jana]

What did you do? setup monitoring for presence/absence of cluster leader using consul_raft_leader metric

What did you expect to see? When external queries to consul cluster fail with HTTP 500 or "No cluster leader" error, we expect to see consul_raft_leader value change from 1 to 0

What did you see instead? Under which circumstances? consul_raft_leader value still remains 1 despite there being obvious issues with cluster health. We can confirm based on logs that show the "No cluster leader" errors, as well as using "consul operator raft list-peers" command

Environment Linux

• consul_exporter version: 0.7.1

• Consul version: Consul v1.8.3

• Prometheus version: N/A

• Prometheus configuration file: N/A

• Logs:

Error getting peers: Failed to retrieve raft configuration: Unexpected response code: 500 (No cluster leader)