faust icon indicating copy to clipboard operation
faust copied to clipboard

Faust with Django connection closed problem

Open MedAziz11 opened this issue 3 months ago • 0 comments

I'm using Faust in combination with a Django backend. Faust is integrated as a Django app but runs in a separate container, similar to how Celery workers operate — except in this case, it's Faust with Kafka.

Inside my Faust agents, I'm performing asynchronous operations using Django's ORM. After the system runs for a while, I begin encountering connection closed errors during ORM operations. This seems related to how database connections are managed in long-lived async processes. any idea how to handle this? logs:

920 | psycopg2.InterfaceError: connection already closed |  
-- | -- | --
  |   | 2025-09-12 09:56:15.920 | django.db.utils.InterfaceError: connection already closed |  
  |   | 2025-09-12 09:56:15.920 | psycopg2.InterfaceError: connection already closed |  
  | 
 | Fields |   | container_name | processor_agent.1.dic96f1ikb3zgicm8nl9l7ih7 |   | filename | /var/log/docker/9cd7bc50abb53e57aabebe6f7f7b71df007ae4be86155ac89313e551c0fda5e3/json.log |   | host | XXXXXXX |   | source | stderr |   | swarm_service | processor_agent |   | swarm_stack | alertify
Fields
  | container_name | processor_agent.1.dic96f1ikb3zgicm8nl9l7ih7
  | filename | /var/log/docker/9cd7bc50abb53e57aabebe6f7f7b71df007ae4be86155ac89313e551c0fda5e3/json.log
  | host | XXXXXXXXX
  | source | stderr
  | swarm_service | processor_agent
  | swarm_stack | xxxx
  |   | 2025-09-12 09:56:15.920 | django.db.utils.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:56:15.920 | psycopg2.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:40:40.671 | elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) |  
  |   | 2025-09-12 09:40:40.671 | urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) |  
  |   | 2025-09-12 09:40:40.671 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:40:40.671 | http.client.RemoteDisconnected: Remote end closed connection without response |  
  |   | 2025-09-12 09:40:40.671 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:00:02.668 | elastic_transport.ConnectionError: Connection error caused by: ProtocolError(('Connection aborted.', RemoteDisconnected('Remote end closed connection without response'))) |  
  |   | 2025-09-12 09:00:02.668 | urllib3.exceptions.ProtocolError: ('Connection aborted.', RemoteDisconnected('Remote end closed connection without response')) |  
  |   | 2025-09-12 09:00:02.668 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:00:02.668 | http.client.RemoteDisconnected: Remote end closed connection without response |  
  |   | 2025-09-12 09:00:02.668 | raise RemoteDisconnected("Remote end closed connection without" |  
  |   | 2025-09-12 09:00:02.428 | django.db.utils.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:00:02.428 | psycopg2.OperationalError: server closed the connection unexpectedly |  
  |   | 2025-09-12 09:00:02.428 | [2025-09-12 10:00:02,374] [7] [ERROR] 🔥 Error processing alert 'XXXXXXXXX': server closed the connection unexpectedly |  
  |   | 2025-09-12 09:00:02.346 | psycopg2.InterfaceError: connection already closed |  
  |   | 2025-09-12 09:00:02.346 | django.db.utils.InterfaceError: connection already closed

MedAziz11 avatar Sep 12 '25 14:09 MedAziz11