python-driver
python-driver copied to clipboard
Memory leak during reconnection storm test
While testing https://github.com/scylladb/python-driver/pull/473 I have found that in scenaior when driver can't properly initialize connections it start eating memory, in my case session endup consuming twice more memory than usual.
Repro: In order to test both PRs I have setup an environment with a cluster of 2 nodes, 16 smp each. And a swarm of client containers, each container holds 200 sessions.
Test scenario:
- Start cluster of 2 nodes, 16 smp each
- Populate the swarm of clients up to 30 clients: 2.1. Start with 0 containers 2.2. Grow the number of containers by 10 2.3. Wait till all shard connections are created 2.4. Repeat until reach target limit
- Restart one of the nodes
- Monitor number of connections in the swarm
You need to run it on scylladb/scylla:2025.2.1 with uninitialized_connections_semaphore_cpu_concurrency high enough to let scylla lag and fail connections during initialization.
Most-likely connections are leaking and stay in memory.
Repro code is attached. test.zip