pals icon indicating copy to clipboard operation
pals copied to clipboard

Intermittent IdleInTransactionSessionTimeout

Open gavinmh opened this issue 4 years ago • 5 comments

Thanks for this useful package. Intermittently, my applications encounter the following exception:

IdleInTransactionSessionTimeout: terminating connection due to idle-in-transaction timeout
SSL connection has been closed unexpectedly

  File "sqlalchemy/engine/base.py", line 1245, in _execute_context
    self.dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 588, in do_execute
    cursor.execute(statement, parameters)
  File "ddtrace/contrib/dbapi/__init__.py", line 99, in execute
    return self._trace_method(self.__wrapped__.execute, self._self_datadog_name, query, {}, query, *args, **kwargs)
  File "ddtrace/contrib/psycopg/patch.py", line 56, in _trace_method
    return super(Psycopg2TracedCursor, self)._trace_method(method, name, resource, extra_tags, *args, **kwargs)
  File "ddtrace/contrib/dbapi/__init__.py", line 69, in _trace_method
    return method(*args, **kwargs)
InternalError: (psycopg2.errors.IdleInTransactionSessionTimeout) terminating connection due to idle-in-transaction timeout
SSL connection has been closed unexpectedly

[SQL: set lock_timeout = %(timeout)s]
[parameters: {'timeout': 10000}]
(Background on this error at: http://sqlalche.me/e/2j85)
  File "flask/app.py", line 2446, in wsgi_app
    response = self.full_dispatch_request()
  File "flask/app.py", line 1951, in full_dispatch_request
    rv = self.handle_user_exception(e)
  File "ddtrace/contrib/flask/helpers.py", line 22, in wrapper
    return func(pin, wrapped, instance, args, kwargs)
  File "ddtrace/contrib/flask/helpers.py", line 31, in wrapper
    return wrapped(*args, **kwargs)
  File "flask/app.py", line 1820, in handle_user_exception
    reraise(exc_type, exc_value, tb)
  File "flask/_compat.py", line 39, in reraise
    raise value
  File "flask/app.py", line 1949, in full_dispatch_request
    rv = self.dispatch_request()
  File "ddtrace/contrib/flask/helpers.py", line 22, in wrapper
    return func(pin, wrapped, instance, args, kwargs)
  File "ddtrace/contrib/flask/patch.py", line 474, in _traced_request
    return wrapped(*args, **kwargs)
  File "flask/app.py", line 1935, in dispatch_request
    return self.view_functions[rule.endpoint](**req.view_args)
  File "ddtrace/contrib/flask/wrappers.py", line 25, in trace_func
    return wrapped(*args, **kwargs)
  File "connexion/decorators/decorator.py", line 48, in wrapper
    response = function(request)
  File "connexion/decorators/uri_parsing.py", line 144, in wrapper
    response = function(request)
  File "connexion/decorators/validation.py", line 384, in wrapper
    return function(request)
  File "connexion/decorators/parameter.py", line 121, in wrapper
    return function(**kwargs)
  File "src/api.py", line 179, in get_premises
    "location_risks": _get_risks(id, cached_only)['risks'],
  File "src/api.py", line 52, in _get_risks
    with lock:
  File "pals/core.py", line 132, in __enter__
    if not self.acquire():
  File "pals/core.py", line 101, in acquire
    self.conn.execute(timeout_sql, timeout=acquire_timeout)
  File "sqlalchemy/engine/base.py", line 982, in execute
    return meth(self, multiparams, params)
  File "sqlalchemy/sql/elements.py", line 293, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "sqlalchemy/engine/base.py", line 1095, in _execute_clauseelement
    ret = self._execute_context(
  File "sqlalchemy/engine/base.py", line 1249, in _execute_context
    self._handle_dbapi_exception(
  File "sqlalchemy/engine/base.py", line 1476, in _handle_dbapi_exception
    util.raise_from_cause(sqlalchemy_exception, exc_info)
  File "sqlalchemy/util/compat.py", line 398, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "sqlalchemy/util/compat.py", line 152, in reraise
    raise value.with_traceback(tb)
  File "sqlalchemy/engine/base.py", line 1245, in _execute_context
    self.dialect.do_execute(
  File "sqlalchemy/engine/default.py", line 588, in do_execute
    cursor.execute(statement, parameters)
  File "ddtrace/contrib/dbapi/__init__.py", line 99, in execute
    return self._trace_method(self.__wrapped__.execute, self._self_datadog_name, query, {}, query, *args, **kwargs)
  File "ddtrace/contrib/psycopg/patch.py", line 56, in _trace_method
    return super(Psycopg2TracedCursor, self)._trace_method(method, name, resource, extra_tags, *args, **kwargs)
  File "ddtrace/contrib/dbapi/__init__.py", line 69, in _trace_method
    return method(*args, **kwargs)

Are connections being left in the "idle in transaction" state? Can these transactions be committed or rolled back?

gavinmh avatar Dec 04 '20 21:12 gavinmh