freeswitch
freeswitch copied to clipboard
signalwire
What is the true session limitation when the resources are plenty?
I am trying to see how many sessions I can create in a ridiculously high end server. I am doing this for capacity planning purpose. This is using Freeswitch 1.10.5 on Debian 10.x edition.
Instance type: AWS c5.9xlarge (36cpu + 72GB)
The test consists of running 3 freeswitches on 3 separate c5.9xlarge instances, switch 1 is acting like agent endpoint, switch 2 is acting like customer endpoint, and switch 3 is conferencing the two calls. All calls are initiated from FS3, towards FS1 and FS2, and then each pair of them are added to a conference on FS3. I was hoping to hit at least 3000 conferences on FS3, which would mean 6000 sessions.
FS1<-------agent side-------->FS3<-------customer side------>FS2
However, even before the CPU usage crosses 20% on FS3, Artoo gets activated.
2022-07-21 18:41:56.216432 [CRIT] switch_core_session.c:1861 Thread Failure! 2022-07-21 18:41:56.216432 [CRIT] switch_core_session.c:1817 LUKE: I'm hit, but not bad. 2022-07-21 18:41:56.216432 [CRIT] switch_core_session.c:1818 LUKE'S VOICE: Artoo, see what you can do with it. Hang on back there.... Green laserfire moves past the beeping little robot as his head turns. After a few beeps and a twist of his mechanical arm, Artoo reduces the max sessions to 1796 thus, saving the switch from certain doom.
It was having nearly 900 conferences (2 calls each) when Artoo got activated. This behavior is very consistent.
Basically, the switch_thread_create() function fails to create anymore threads even before reaching 2000 threads. I have done all the recommended ulimit changes mentioned in this link. The core.db is on ram disk too. The rtp range is set to 20000 ports.
In the switch.conf:
<param name="max-sessions" value="10000"/> <!--Most channels to create per second --> <param name="sessions-per-second" value="500"/>
These settings are present in all the 3 freeswitches.
So I am not sure why switch_thread_create() is failing at such a low load. It is definitely not CPU or memory. There is no crash. After this point I have to drop calls in order to make new calls.
