examples/http_client.cc fails to GET content larger than 4096 bytes including header

[handrews42]

Consider the following server code:

#include <sstream>

#include <pistache/http.h>
#include <pistache/router.h>
#include <pistache/endpoint.h>

class EchoServer {
public:
  EchoServer(Pistache::Address addr)
    : httpEndpoint(std::make_shared<Pistache::Http::Endpoint>(addr)) {}

  void init(size_t thr = 2) {
    auto opts = Pistache::Http::Endpoint::options()
      .threads(thr);
    httpEndpoint->init(opts);
    setup();
  }

  void start() {
    httpEndpoint->setHandler(router.handler());
    httpEndpoint->serve();
  }

private:
  void setup() {
    Pistache::Rest::Routes::Get(router, "/:value/:count", Pistache::Rest::Routes::bind(&EchoServer::doEcho, this));
  }

  void doEcho(const Pistache::Rest::Request& request, Pistache::Http::ResponseWriter response) {
    std::string value = request.param(":value").as<std::string>();
    int count = request.param(":count").as<int>();

    std::stringstream ss;
    for (int i=0; i<count; ++i) { ss << value; }
    response.send(Pistache::Http::Code::Ok, ss.str());
  }

  std::shared_ptr<Pistache::Http::Endpoint> httpEndpoint;
  Pistache::Rest::Router router;
};

int main(int argc, char *argv[]) {
  Pistache::Port port(9080);
  Pistache::Address addr(Pistache::Ipv4::any(), port);

  EchoServer echo(addr);

  echo.init();
  echo.start();
}

It is adapted (simplified) from the example/rest_server.cc code in this repo. When run and hit from a browser, it will repeat the ":value" field ":count" times as expected. When using the examples/http_client.cc code, however, it works only when the GET response is less than ~4096 characters. This can be verified by running the http_client.cc program with the following while the server above is running on localhost.

This works as anticipated: http_client http://localhost:9080/f/4036

This fails to retrieve a response: http_client http://localhost:9080/f/4037

Both of these URL's work when accessing from a web browser. I have also verified this using other web pages which return more than 4KiB. If the response is larger than 4096 bytes, the client fails to process it.

I attempted to change MaxBuffer in include/pistache/config.h to 16MB, but that resulted in a seg fault. I didn't try any smaller values.

[dennisjenkins75]

As mentioned in many other bugs in this project, the pistache client is horribly broken. I wish people would search through the bug list before filing duplicate bugs.

PRs to fix the http client are welcome. PRs to remove it are also welcome. I recommend that you use libcurl or some other http client. If you need the promise/callback features of the pistache client, then you or someone will need to submit PRs to fix bugs and add support.

The original author of pistache only needed very small request and response sizes, so that is all that he implemented. We modified the server side so that we can override the 4k max buffer when starting up the server thread. No such change has been made to the client yet.

[hyperxor]

@handrews42 Hello, I have made several changes (they are in master) to fix this bug. Would it be possible for you to check it? @dennisjenkins75