python-server-sdk icon indicating copy to clipboard operation
python-server-sdk copied to clipboard

Large volume of logs reading `Unhandled exception in event processor`

Open hwong557 opened this issue 1 year ago • 0 comments

Is this a support request? No

Describe the bug A very large volume error logs reading Unhandled exception in event processor occured at a rate of about 97k logs/min until our service was killed. This occured on a Sunday evening when no changes in infra happened.

We run our FastAPI app on a gunicorn server.

To reproduce We have been unable to reproduce the log.

Expected behavior Error to not have happened.

Logs Below is redacted structured log output. You can see the stack trace in the log.

{
	"message": "Unhandled exception in event processor",
	"attributes": {
		"exception": [
			{
				"is_cause": false,
				"frames": [
					{
						"filename": "/usr/local/lib/python3.11/site-packages/ldclient/impl/events/event_processor.py",
						"lineno": 358,
						"line": "",
						"name": "_run_main_loop"
					},
					{
						"filename": "/usr/local/lib/python3.11/queue.py",
						"lineno": 171,
						"line": "",
						"name": "get"
					},
					{
						"filename": "/usr/local/lib/python3.11/threading.py",
						"lineno": 323,
						"line": "",
						"name": "wait"
					}
				],
				"exc_type": "AttributeError",
				"exc_value": "'Condition' object has no attribute 'gunicorn'"
			}
		],
		"process": 36,
		"level": "error",
		"logger": "ldclient.util",
		"pathname": "/usr/local/lib/python3.11/site-packages/ldclient/impl/events/event_processor.py",
		"lineno": 377,
		"stream": "stderr",
		"time": "2024-05-20T03:54:53.9583652Z"
	}
}

SDK version

launchdarkly-eventsource==1.1.1
launchdarkly-server-sdk==9.2.1

Language version, developer tools Python 3.11.

OS/platform Debian bookworm.

Additional context Add any other context about the problem here.

hwong557 avatar May 20 '24 23:05 hwong557