ocaml-cohttp icon indicating copy to clipboard operation
ocaml-cohttp copied to clipboard

Memory leak in `Client.read_response`(?)

Open jrochel opened this issue 5 years ago • 1 comments

This is not necessarily to be classified as a memory leak, but it surely looked like one. I observed a continuous rise of memory consumption in a routine making frequent HTTP requests. A tiny bit of memory was being allocated (and not freed) with each request. Running a major GC would free that memory. This could be either achieved by running GC.major () now and then or by tuning the run-time, e.g. using OCAMLRUNPARAM='M=15'. The piece of code consuming the memory was Gc.finalise gcfn stream; in Client.read_response. Commenting it out would keep memory consumption constant.

I don't know which of the following conclusions I should draw from this:

  • There is something wrong with the method that is used to close the stream.
  • Everything is fine. Sometimes memory consumption rises for a while.
  • Everything is fine. Sometimes one needs to tune the GC.
  • The GC needs better heuristics; the major GC should run more often.
  • The above example should be handled by a minor GC.

BTW, I'm talking about an increase of about 25 M in the course of 3 minutes. That's when a major GC kicks in. This might seem negligible but in conjunction with https://github.com/ocsigen/lwt/issues/762 things looked quite alarming.

jrochel avatar Feb 10 '20 18:02 jrochel

Hi @jrochel can you tell me if this is still a problem with cohttp.4.0.0 or the current master?

mseri avatar Apr 17 '21 19:04 mseri