sql_exporter does not recover from killed connection

[0xErnie]

Even if the MySQL database is available again, the exporter still indicates that the database would be offline. Restarting the exporter solves the problem.

I0609 02:55:40.728795   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] Error 1927: Connection was killed
[mysql] 2021/06/09 02:55:55 packets.go:36: unexpected EOF
I0609 02:55:55.728435   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] invalid connection
[mysql] 2021/06/09 02:56:10 connection_go18.go:22: invalid connection
I0617 02:55:25.729805   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] Error 1927: Connection was killed
I0617 02:55:40.729637   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection
I0617 02:55:55.729257   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection
...
I0617 09:38:10.729547   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection
I0617 09:38:25.729122   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection
I0617 09:38:40.730307   25695 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection

Our configuration:

---
# Global defaults.
global:
  scrape_timeout_offset: 500ms
  min_interval: 0s
  max_connections: 3
  max_idle_connections: 3

target:
  data_source_name: {{ item.url }}

  collectors: [managed_mysql]

collector_files:
  - managed_mysql_collector.yml

---
collector_name: managed_mysql

metrics:
  - metric_name: mysql_stats
    type: gauge
    help: 'Get lots of stats'
    key_labels:
      - Variable_name
    values: [Value]
    query: |
      SHOW status
      WHERE variable_name
      IN ('threads_connected',
          'Aborted_clients',
          'Aborted_connects',
          'Access_denied_errors',
          'Acl_users',
          'Bytes_received',
          'Bytes_sent',
          'Connections',
          'Max_used_connections',
          'Memory_used',
          'Uptime',
          'Innodb_data_read',
          'Innodb_data_reads',
          'Innodb_data_writes',
          'Innodb_data_written'
          )

[0xErnie]

I decided to switch to the fork burningalchemist/sql_exporter which has updated multiple dependencies.

[rgardelein]

I am experiencing the same problem with the sql_exporter, where the exporter needs to be restarted in case of network connection problems to the database. Anybody has a working solution for this problem?

[rgardelein]

Still suffering from the same issue. The exporter is not recovering from DB connectivity problems. Anybody has an idea on how to resolve?

I0513 23:50:20.063691 1 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection I0513 23:51:20.061513 1 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection I0513 23:52:20.062154 1 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection I0513 23:53:20.064057 1 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection I0513 23:54:20.063563 1 promhttp.go:38] Error gathering metrics: [from Gatherer #1] bad connection

[0xErnie]

@rgardelein Have you tried my solution?

[rgardelein]

Hello 0xErnie, Thank you for your reply. I have not tried your solution by switching to the burningalchemist/sql_exporter. Is this solution fully upward compatible? What is your experience with it? rgardelein