History feature is memory hungry

[Speedkore]

Could you make please a setting to disable history or limit the number of entries or disable saving response data?

I'm testing an api that returns 10MB jsons and in every request i see how my memory goes lower and lower :D ( I don't want to hit the "clean up history" button every now and then)

Btw this plugin is amazingly simple, fast and easy to use, great work!!

[frigus02]

Thanks for the report. This seems to be similar to what a reviewer was reporting (#58). And it does sound quite annoying.

I see 2 issues here:

1. Memory usage should not increase when making multiple requests. I would expect only the currently visible request/response to be fully in memory. 👉 I think that may be a bug. I will look into this.

2. This can quickly take up loads of disk space (and potentially make RESTer slow). 👉 Seems like a setting to not store payloads larger than X KB could help. I wonder if that should apply only to response bodies or if other parts like response headers, request bodies and headers are also affected.

[frigus02]

I can't quite seem to reproduce it. Here is what I tried:

• Start a local test server, which generates a large JSON response. You can test it by:
  • Cloning this repo
  • Checking out the branch memory-leak
  • Running node ./tools/dev-utils/server.js
  • Making a request in RESTer to GET http://localhost:7373/large
• I monitored RESTer in both about:performance and the Windows task manager.
• I made about 10 requests to the /large endpoint in RESTer.
• I cleaned up the history in RESTer and repeated the requests.
• I restarted Firefox and repeated the requests.

Results:

• In about:performance RESTer was always between 50MB and 200MB
• In the Windows task manager I could see Firefox' memory grow quickly from ~1GB after a fresh start to ~3.5GB. Every request I made in RESTer increased the memory by at least 500MB. However memory usage didn't go higher than that and actually went back down automatically after a few seconds.

So apparently it's not that simple. 🙂

Can you describe how your setup differs from my tests?

[Speedkore]

i will try your test and compare it with mine asap :thumbsup:

[Speedkore]

I tested your request, i can do only one, i think i need more memory and a better pc because parsing the json takes some minutes, and i'm already using a lot of ram having many big open apps already so i notice very fast when something grows in memory.

And you are right, the memory frees alone after a while, there's no need to do the clear history, but it is what it is, browsers eat a lot of ram, not much you can do.

[frigus02]

Okay, so the good thing is that there doesn't seem to be a memory leak. However adding more RAM shouldn't be the solution, I think.

I'm going to experiment a bit. If it's JSON parsing and maybe highlighting that's the issue, maybe I can turn this off for large responses.

[frigus02]

It looks like the high memory consumption comes mainly from rendering the response body in Ace. Having "Wrap" turned on makes it a lot worse, so you may want to try turning that off.

I also found that RESTer could do pretty printing slightly smarter. At the moment it:

• renders the normal body before switching to the pretty printed version
• pretty prints even if the option is turned off (it just doesn't show the result)

I improved those points. However I found that pretty printing JSON is really fast and doesn't seem to use a lot of memory. If the response body contains very long lines pretty printing actually seems to help.

The last thing I'm going to look into is turning "Wrap" off automatically for large responses. Then I'm going to release an update with the improvements.

[Speedkore]

Those things will surely help, awesome!

[frigus02]

Version 3.11.2 should be available as an update shortly.

I ended up not turning "Wrap" off automatically because long lines are quite hard to detect. But I added a warning for large responses, that turning off "Wrap" can improve the performance.

I have one last rather drastic idea on how improve memory usage. I could completely hide large response bodies. Would that be helpful for you? I assume it could be a bit useless because the response body is generally the most important thing 🤔

[Speedkore]

Yeah that's the point of all, view the response haha

Version 3.11.2 should be available as an update shortly. :+1: :+1: :+1:

[frigus02]

That's what I thought 😄. I thought there may be a slight chance you just want to see the status code or headers.

I don't think there is any other thing wrong with RESTer. But memory consumption get still get very high, so I'm going to leave this issue open. Maybe I find another, less memory hungry way, to render the response body in the future.