Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

consul_raft_leader does not reflect correct state of leader #208

Open
Abhimanyu-Jana opened this issue Jul 30, 2021 · 0 comments
Open

consul_raft_leader does not reflect correct state of leader #208

Abhimanyu-Jana opened this issue Jul 30, 2021 · 0 comments

Comments

@Abhimanyu-Jana
Copy link

What did you do?
setup monitoring for presence/absence of cluster leader using consul_raft_leader metric

What did you expect to see?
When external queries to consul cluster fail with HTTP 500 or "No cluster leader" error, we expect to see consul_raft_leader value change from 1 to 0

What did you see instead? Under which circumstances?
consul_raft_leader value still remains 1 despite there being obvious issues with cluster health. We can confirm based on logs that show the "No cluster leader" errors, as well as using "consul operator raft list-peers" command

Environment
Linux

  • consul_exporter version:
    0.7.1

  • Consul version:
    Consul v1.8.3

  • Prometheus version:
    N/A

  • Prometheus configuration file:
    N/A

  • Logs:

Error getting peers: Failed to retrieve raft configuration: Unexpected response code: 500 (No cluster leader)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant