HTTPClient code does not collect header names

[MikeyMoMo]

Related area

WiFi

Hardware specification

ESP32

Is your feature request related to a problem?

I finally got header fetch working. I find a problem with it.

Firstly, this feature should have an example. I can provide one since I finally got it working on my fourth attempt.

Second:

1. One has to know the name of the header(s) and list them in the C code to be able to get them.
2. The http.headers() call does not report the actual number of headers in the returned data, it simply reports the number of names you entered into the headerKeys array. I already know that number, I had to enter them. It should return the number of headers that are available so they can be iterated through.
3. The author uses size_t and the current Arduino compiler release has a problem with that definition. I forgot the error message but it is repeatable, for the most part. Sometimes it care, sometimes it does not care. It would be easier on us programmers if you used a long integer or even just integer. There will not be over 2 billion headers in one file. size_t is surely overkill and an obscure variable type that confuses the compiler.
4. headerKeysCount already has the same number that http.headers() returns. Redundant and, therefore, useless. This needs correction to return the actual count of found headers. In the data I receive, there are 4 headers but .headers returns 1 because I listed only 1 in the array.
5. Let Firefox be your guide. It shows all four returned headers. That's how I verified they are there and verified the name of the header I needed.

Here is the code I ended up with (after 3 wrong/incomplete AI responses and then more in-depth digging through web pages):

const char *headerKeys[] = {"x-ratelimit-remaining-month"};  // The only header I care about.
const size_t headerKeysCount = sizeof(headerKeys) / sizeof(headerKeys[0]);  // Returns 1
http.collectHeaders(headerKeys, headerKeysCount);

if (iHttpResponseCode == 200) {  // 200 is goodness!

    // Print all headers
    int headerCount = http.headers();  // Simply a count of elements of the headerKeys array.  Returns 1
    //      Serial.printf("\r\n%i HTTP header(s) sought:\r\n", headerCount);
    for (int i = 0; i < headerCount; i++) {
      //        Serial.println(http.header((size_t) 0));  // size_t is required to be coded!
      sTemp = http.header((size_t) 0);  // size_t is required to be coded!
      sTemp = sTemp.substring(0, sTemp.lastIndexOf(":"));
      //        Serial.printf("%s: %s gave %s\r\n",
      //                      http.headerName(i).c_str(),
      //                      http.header(i).c_str(), sTemp);
      Serial.printf("There are %s /*%i*/ XRate calls left for this key"
                    " this month.\r\n", sTemp/*, sTemp.toInt()*/);
}

Describe the solution you'd like

See above.

Return all headers you find. Return the count of all headers you find for iteration through them. Don't require the user to specify the header name in advance. Collect all of them. Create a working example. (I have one if you want to start from there)

Describe alternatives you've considered

None. There are no alternatives. I use HTTPClient.

Additional context

No response

I have checked existing list of Feature requests and the Contribution Guide

• [X] I confirm I have checked existing list of Feature requests and Contribution Guide.

[MikeyMoMo]

I forgot to say thank you for the code. I use it daily.

I am still looking for a way to set a timeout that works. There are lots of answers on the web but none I have found work.

[me-no-dev]

we do not collect all headers on purpose. They do take quite a bit of memory in many cases. We could look into a way to say specifically to collect them all. Would that work for you?

[silencify]

Thank you for the code @MikeyMoMo . I was able to modify your code to get header. Below is my code:

#include <HTTPClient.h>

HTTPClient http;
http.begin(baseUrl);
http.addHeader("Content-Type", "application/json");

const char *headerKeys[] = {"Server"};  // header that I need
const size_t headerKeysCount = sizeof(headerKeys) / sizeof(headerKeys[0]);
http.collectHeaders(headerKeys, headerKeysCount);

int httpCode = http.GET();

if (httpCode == 200) {
    String header = http.header("Server");  // header that I need
    Serial.print("header: ")
    Serial.println(header);
    String response = http.getString();
    Serial.println(response);
} else {
    Serial.println("Error on HTTP request");
}
    
http.end();

[MikeyMoMo]

Yes, this location only sends out 4 headers so that should not tax the memory. All features like this should be optional so that would be great. It is a low priority but something that would be nice and fit in with other "browsers". Like Firefox when it shows all received headers. Thanks. Reliable code. I fetch weather 3 times per hour and it has run for many months.