[WIRE136] Network redesign

[andreagilardoni]

As the title states the objective of this PR is the redesign of the network stack of the portenta C33. This PR aims to solve multiple issues that affected reliability, performance and usability of networking.

The main issues addressed are the follwing:

• Unreliable TCP streams over both WiFi and Ethernet
• Bad performances of Ethernet drivers
• Bad Performances of WiFi drivers
• Memory leaks in the code on different levels, starting from missing destructors of Client classes
• incorrect management of lwip pbufs that lead to missing chunks in TCP communication

This issue needs https://github.com/arduino/ArduinoCore-API/pull/218 in order to completely fix memory leaking when using a class that is derived from arduino::Client.

Since this work started as a task to address Ethernet bad performances and behavior it was mainly focused on the ethernet improvement. We were able to achieve a speed of ~80Mbit/s in Rx with some fine tuning of the libraries (this require high ram usage) by means of Rx zero copy capabilities of Ra6m5 controller. Wifi drivers were not taken into consideration that deeply, but there is room for improvement, i.e. by reducing the number of memcpy performed on buffers.

Furthermore some aspects of the integration of lwip have been improved:

• now a WiFi Client runs exclusively on WiFi interface, the same is valid with Ethernet
• the dhcp client interface has been simplified to leverage more on lwip implementation
• the dns integration has been simplified
• the code has been designed to run mostly in a non-blocking way, in order to avoid starvation in a system that doesn't support threads

This issue is currently on draft, since there are still some imperfections, which are addressed in TODO and FIXME comments. Some will be addressed before this PR is merged and others are going to provide a path to follow for future improvements.

[andreagilardoni]

@JAndrassy can you please take a look at this?

This also includes the changes you did in https://github.com/arduino/ArduinoCore-renesas/pull/209 and addresses the issues related to https://github.com/arduino/ArduinoCore-renesas/pull/210

[JAndrassy]

sorry. this are too many changes at once. and some of them are just formatting changes which makes it even harder to see what is happening. I tried to go through individual commits, but that didn't help me either to see what is really changing.

[andreagilardoni]

I will try to improve commits before removing this from draft. The problem with this PR is that the issue of reliability of network was diffused in different levels of the code and in the way I approached it, it became hard (for me) to split it in different PRs.

[JAndrassy]

In my opinion as first don't change the code style of otherwise unchanged lines. and the rework of the Ethernet library is I think a separate thing.

[pennam]

@andreagilardoni Arduino_ConnectionHandler library fails to build with this branch.

class CEth' has no member named 'linkStatus'
class CEth' has no member named 'disconnect'

[Eduardo-Lopes]

Hi! I tried to run the WebServer example of "network-redesign" branch. There is some issue with DHCP: Appears this message on serial: "Failed to configure Ethernet using DHCP"

On 1.0.5 version, it works.

Thanks for your attention

[JAndrassy]

@andreagilardoni my NetAPiHelpers library's examples are compatibility tests for Arduino Networking API. the library is in library manager or here https://github.com/JAndrassy/NetApiHelpers

also I wrote a guide to Arduino Networking API for library developers. it is here https://github.com/JAndrassy/Arduino-Networking-API/blob/main/ArduinoNetAPIDev.md

[andreagilardoni]

@Eduardo-Lopes I think I solved the issues you were describing, if you can confirm me that.

@JAndrassy I will for sure take a look at your work and take it into consideration for further deveplment, thanks for your work!

[Eduardo-Lopes]

Hi @andreagilardoni, unfortunately, the problem persists. Also, the problem is on Ethernet and DHCP, not in EthernetServer or EthernetClient.

[andreagilardoni]

@Eduardo-Lopes sorry, I forgot to add a part of the fix

[Eduardo-Lopes]

@andreagilardoni the DHCP is solved, but still there is connection loss when I stress test.

[andreagilardoni]

@Eduardo-Lopes how are you performing stress tests? So that can better address the issue

[Eduardo-Lopes]

I just call a Python script that loops requesting the WebServer on C33. I added the code on the issue: https://github.com/arduino/ArduinoCore-renesas/issues/240#issue-2081911859

[andreagilardoni]

This will take some more time to debug, I think there is a memory leak somewhere I am missing.

[javos65]

This problem of instable WebServer is been around for 2-3 years, still not solved? It's typical for Arduoino with ESP32 wifi modules, like on the wifiNINA of UBlox. Even out of the box examples like SimpleWebserverWifi does not work stable....