Behaviour of CONFIG_NET_ALLOC_CONNS

[fjpanag]

A while back I studied the implementation of CONFIG_NET_ALLOC_CONNS, and I saw some things that seemed strange to me.

I would like to discuss my thoughts, being motivated by #6956.

I see that when CONFIG_NET_ALLOC_CONNS is enabled, arrays like g_cbprealloc[] are not defined. Rather, the connection structures are dynamically allocated, as expected.

However, I see that:

• When new connection structure is needed, a batch of them is allocated. Not only one. For example in tcp_conn.c line 559.
• When the connection is closed, the structure is returned in the "free" list instead of it being deallocated.

I believe that this is not correct.
When you only need one single new connection, the memory allocated may be much more than actually needed.
If the system experiences a load-spike, creating many new connections, it will never be able to return to "normal". Memory used for the connections is lost forever, multiple connections may not be needed ever again.

In one of our cases this is more prominent.
During system boot many connections are opened towards various services. When everything is ready, the system keeps only one connection open for the rest of its life-time.

---

I propose the following changes:

• Allocate only one structure when needed.
• Ensure that when finished, the connection is actually de-allocated properly.
• Maybe keep a small (and configurable) pool of structs always allocated (either from the heap or BSS), to reduce heap fragmentation, and enhance performance. This will be useful for example, in cases of multiple successive calls to netlib_*() functions.

[fjpanag]

Personally I think I would prefer us to ditch the CONFIG_NET_ALLOC_CONNS option completely.

Always have a poll of statically allocated structs, for example using options like CONFIG_NET_TCP_CONNS, and when this pool is exhausted, proceed to heap allocations.

A hybrid approach. Like CONFIG_PREALLOC_TIMERS.

[xiaoxiang781216]

If you like allocate/free every time when the pool is exhausted and don't care about the fragmentation. We can simply do this special action by checking CONFIG_NET_ALLOC_CONNS == 1

[fjpanag]

If you like allocate/free every time when the pool is exhausted and don't care about the fragmentation. We can simply do this special action by checking CONFIG_NET_ALLOC_CONNS == 1

No it is not the same. In this case, when you finish using the connection it is returned to the free list, it is not deallocated with a proper call to free().

Thus your system ends up using memory equal to its maximum number of connections it had seen, regardless of the current needs.

Consider for example a case were 20 connections are needed for 5 mins and then only 1 for the rest of the operation. In this case, 20 connections will always be reserved from the heap, forever.

[xiaoxiang781216]

I mean you can kmm_free directly in case of ONFIG_NET_ALLOC_CONNS == 1 not put to the pool.

[fjpanag]

As work is being done on this, I am taking note of the connections that need improvement:

• [x] Bluetooth (g_bluetooth_connections)
• [x] Devif (g_cbprealloc)
• [x] ICMP (g_icmp_connections)
• [x] ICMPv6 (g_icmpv6_connections)
• [x] IEEE802154 (g_ieee802154_connections)
• [x] Netlink (g_netlink_connections)
• [x] Pkt (g_pkt_connections)
• [x] TCP (g_tcp_connections)
• [x] UDP (g_udp_connections)
• [x] Usrsock (g_usrsock_connections)
• [x] CAN (g_can_connections)