Overload handling: low heap and high number of open connections

[amkrk]

Stability improvements to the server, allowing to survive momentary overload e.g. when serving more the 4 files on handling a few clients.

Overload control thresholds: Level1: (max connections exceeded): send 429 with header "Retry-After" set to 1 second Level2: (low free Heep): send pure 500 (no extra info added into reponse header) Level3: (very low free help or if can not create AsyncWebServerRequest object - request too big): immediate drop underlying TCP connection

It's been tested on ESP8266 (built on Arduino Windows). All above strategies looks to work well. Client (web browser) re-tries http request after getting 429 or 500. This allows to serve html projects with more than ~4 (html/CSS/js/...) files being requested simultaneously on ESP8266 (10 files in case of my html project). Closing connection causes a web browser to finish loading the webpage without waiting forever for a request which has been shed due to the overload.

Overall: an application started to work stable now :) No spurious resets even after 1 day and while working with 8 browsers connected and running an application (web API calls every 1 sec from each browser in parallel with loading webpage's body from other clients).

ESP32: MAX_NUM_OF_HTTP_REQUESTS - this may likely be set higher (more RAM just for the server), but I was not testing this on ESP32.

---

Next steps (likely to be done in a separate commit): check against heap defragmentation (test allocation and free of one a few kB block) before processing http request. Applicaion controlled reset if the test allocation fails. Calling user's callback function before the reset, allowing to send e.g. good bye / I'm rebooting msg through a serial line into coo-working MCU.

[zekageri]

I would love to see this implemented. :(

[amkrk]

"This pull request can be automatically merged by project collaborators Only those with write access to this repository can merge pull requests."

I'm not in the group, so those who are: go ahead with the merge. I'm not sure about regression testing accross several platforms, i.e. how it works in the project ...

[blazoncek]

I am voting +1 for this PR to be merged.

[GeorgeFlorian]

@amkrk would this require changes to an implementation ? Does it change how we write the server code ? Or is it doing it all under the hood ?

I will test these changes on an ESP32 because I have a medium sized project that's being served by this library and it sometimes hangs.

[MetriciRO]

@amkrk These changes make my project not load all its css and js files:

The error is a HTTP 429 Too Many Requests.

It also slows down the interface quite a lot.

server.serveStatic("/settings", SPIFFS, "/settings.html");
server.serveStatic("/dashboard", SPIFFS, "/index.html");
server.serveStatic("/user", SPIFFS, "/user.html");
server.serveStatic("/", SPIFFS, "/").setCacheControl("max-age=600");

[zekageri]

I just wanted to try. Iam sad to hear that

[amkrk]

I'm sorry to hear that. I guess (as stated in a description) the ovld controll feature needs a calibration on esp32. I've just setup those limits for ESP8266 (very small on RAM, etc.), which may triger the functionality prematurely on ESP32.

In particular: #define MAX_NUM_OF_HTTP_REQUESTS 3

is used by the code:

if(svr->numOfRequests > MAX_NUM_OF_HTTP_REQUESTS){
  AsyncWebServerResponse *response = r->beginResponse(429);
 

Dear @GeorgeFlorian & @MetriciRO , would it be possible for you to experimentally determine proper number for ESP32 ? I do not have the chip, so can not do it ... Then you would need to #ifdef the number and set it for each of these platforms separatelly in src/WebServer.cpp .