Increased memory usage since 4.10.0

[MattRM-M]

Describe the bug I recently switched from version 4.9.4 to the latest version and noticed that the heap usage over runtime in my app increased from ~200MB to ~550MB. After checking different versions, I found that the memory increase started in version 4.10.0.

To Reproduce The test app I created just exposes POST/DELETE/GET/PUT on a number of paths and logs the current memory usage using process.memoryUsage when it receives one of those requests on each path. It also logs memory usage every 10 seconds. I ran a series of tests where I sent a REST request on each available request type/path combination, waited a minute, then sent the same set of requests. I ran this test under for different versions of express-openapi-validator with 16 exposed path/request combinations and with 160 exposed path/request combinations.

I ran my tests on Cent OS 7 and I saw increases of the same magnitude while running an app in docker container based on rhel7.

Actual behavior For both 4.9.5 and 4.10.0, I saw significant memory allocation on receiving the first request on a path/request combination, but not on the second. According to process.memoryUsage(), the increases in memory I saw as I scaled up the test app were from the Heap usage and the Resident Set Size. The memory allocation 4.10.0 allocated more memory than 4.9.5 as detailed below:

4.9.5 with 16 path/request combinations, between right before the first REST request and after the last: Heap usage saw an increase of ~1090K RSS saw an increase of ~4195K

4.9.5 with 160 path/request combinations, between right before the first REST request and after the last: Heap usage saw an increase of ~8540K RSS saw an increase of ~20239K

4.10.0 with 16 path/request combinations, between right before the first REST request and after the last: Heap usage saw an increase of ~3164K RSS saw an increase of ~7082K

4.10.0 with 160 path/request combinations, between right before the first REST request and after the last: Heap usage saw an increase of ~15011K RSS saw an increase of ~35201K

Examples and context I have attached a scaled down version of the test app I used. Attached code consists of app.js, example.js, and swagger.yaml To scale it up to more paths, I added a few more controller files and added new path ids to both the swagger.yaml and to the controller files. All request functions performed the same logic.

app.zip

[cdimascio]

@MattRM-M thanks for this. I will have a look at the app.zip.

[cdimascio]

@MattRM-M Can you provide some details with respect to how you did this profiling. I'm interested in recreating this locally

[MattRM-M]

For my complete app, I had used Grafana in conjunction with running my application as a docker container based off of RHEL 7 in OpenShift.
For the test app, I ran it using the basic 'node app.js' command using node version 12.8.4 but I have observed the problem in node 14. I ran the test app in a CentOS 7 virtual machine. I used calls to process.memoryUsage(), which occurred on a 10 second loop, in order to output the memory statistics. The actual test I ran was this: I would start the test app, wait a minute seconds to ensure a baseline memory usage, then use Postman to send a single REST request on each available endpoint. I used the collection feature in Postman to send all the REST requests over a short period of time. Then I waited a minute for the memory usage to flatline and collected the output. Once I had the output, I compared the memory usage statistics right before the first REST request to the statistics at the end of the test. In order to change the scale of the test, I would just copy the existing endpoints and functions and paste them with new ids to create new REST endpoints. I also created 3 other controller files with similar endpoints. All of the endpoints just logged that they were requested to console.log() and then called process.memoryUsage(). The two main tests I ran on each version were a test with 16 total endpoints across 4 files and a test with 160 total endpoints across 4 files. I also ran tests where I started the app, waited a minute, sent all the REST requests, waited a minute, sent the same set of REST requests, then collected data. However, I did not see an increase in memory usage on the second set of REST requests; just on the first set.