newrelic-airflow-plugin icon indicating copy to clipboard operation
newrelic-airflow-plugin copied to clipboard

certificate verify failed with Airflow 1.10.15

Open rinzool opened this issue 2 years ago • 2 comments

Issue

In my company we are facing an issue with the plugin. We are running Airflow with docker (with a custom image), in version 1.10.15. After installing the plugin, we see in scheduler logs: urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

We settle only two environment variables:

  • NEW_RELIC_INSERT_KEY=***
  • NEW_RELIC_HOST=metric-api.eu.newrelic.com

How to reproduce

The issue can be reproduce locally by pulling public image:

docker run -e NEW_RELIC_HOST=metric-api.eu.newrelic.com -e NEW_RELIC_INSERT_KEY=***  -it apache/airflow:1.10.15 bash

And on the image:

pip install newrelic-airflow-plugin
airflow scheduler

Which directly throw the following error: urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

Full stacktrace
[2022-04-22 14:59:23,560] {harvester.py:83} ERROR - New Relic send_batch failed with an exception.
Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 488, in wrap_socket
    cnx.do_handshake()
  File "/home/airflow/.local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1825, in do_handshake
    self._raise_ssl_error(self._ssl, result)
  File "/home/airflow/.local/lib/python3.6/site-packages/OpenSSL/SSL.py", line 1563, in _raise_ssl_error
    _raise_current_error()
  File "/home/airflow/.local/lib/python3.6/site-packages/OpenSSL/_util.py", line 57, in exception_from_error_queue
    raise exception_type(errors)
OpenSSL.SSL.Error: [('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')]

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 494, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
ssl.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/home/airflow/.local/lib/python3.6/site-packages/newrelic_telemetry_sdk/harvester.py", line 75, in _send
    response = self.client.send_batch(*flush_result)
  File "/home/airflow/.local/lib/python3.6/site-packages/newrelic_telemetry_sdk/client.py", line 236, in send_batch
    return self._pool.urlopen("POST", self.PATH, body=payload, headers=headers)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 727, in urlopen
    method, url, error=e, _pool=self, _stacktrace=sys.exc_info()[2]
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/retry.py", line 386, in increment
    raise six.reraise(type(error), error, _stacktrace)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/packages/six.py", line 734, in reraise
    raise value.with_traceback(tb)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 677, in urlopen
    chunked=chunked,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 381, in _make_request
    self._validate_conn(conn)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connectionpool.py", line 978, in _validate_conn
    conn.connect()
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/connection.py", line 371, in connect
    ssl_context=context,
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/util/ssl_.py", line 386, in ssl_wrap_socket
    return context.wrap_socket(sock, server_hostname=server_hostname)
  File "/home/airflow/.local/lib/python3.6/site-packages/urllib3/contrib/pyopenssl.py", line 494, in wrap_socket
    raise ssl.SSLError("bad handshake: %r" % e)
urllib3.exceptions.SSLError: ("bad handshake: Error([('SSL routines', 'tls_process_server_certificate', 'certificate verify failed')],)",)

rinzool avatar Apr 22 '22 15:04 rinzool