zen
zen copied to clipboard
anfragment
The memory usage is too high, about the same as that of browsers and online games
Description
Version
0.8.0
Operating System
WINDOWS 10
Steps to Reproduce
No response
Additional Context
No response
Hello, @2372281891! Thanks for the report.
Marking this as help wanted—I'd appreciate it if someone could run tests using Go's memory profiler to help identify whether there's a memory leak and/or determine which data structures consume the most memory.
To whoever decides to take this up in the future: Here's a memory profile of Zen after initialization with the filter lists (click on the image to get the full view):
The vast majority is consumed by our primary data structures, ruletree and node, which represent the filter list rules as a trie. They're far from being written in a memory-efficient manner, so there's a potential for improvement.
I am working on this issue on a fork https://github.com/ilovelinabell/zen-desktop but how did you guys do the profiling? I think I have shrunk the mem usage a bit but want to quantitatively compare. If you guys can pull my branch and run the profiler, that could also be good. Thanks
@ilovelinabell Excited to see what comes out of it! The profiling can be done with Go's built-in pprof, this guide looks good enough to follow: https://www.codereliant.io/p/memory-leaks-with-pprof
Thanks for the reply. I am not entirely sure how to use pprof, I will look into it soon. For now, here is a screenshot of my current mem usage.
I have the following filterlists active:
I’m unfortunately unsure whether we can get to <100 MB since we are essentially running an entire HTTP proxy to man-in-the-middle stop ads from being served. You can look at mitmproxy as another example of a HTTP proxy service.
We will keep thinking about optimizations to lower memory usage, however. Thanks
@ilovelinabell If you're up for it, I believe rewriting RuleTree as a compressed trie will help massively with memory consumption
@anfragment when you have time can you run your profiler again on the current application to see what it is reporting for the mem usage per function? I want to see what the breakdown is
The overwhelming majority is still consumed by RuleTree:
(click to get the full image)
I'd note that the memory profiler reports a figure significantly lower than what's shown by the OS. It's just a guess at this point, but I suspect heap fragmentation might be the reason. In addition to compressing the trie, switching to an array-based structure (where nodes are stored in a contiguous array and reference their children by index rather than by pointer) might also yield a significant improvement.
@ilovelinabell fyi, you can reproduce the profiling setup very easily - I've shared the code on anfragment/mem. Once you start the app and initialize the proxy/filter, run:
$ go tool pprof http://localhost:6060/debug/pprof/heap
Fetching profile over HTTP from http://localhost:6060/debug/pprof/heap
Saved profile in /Users/anfragment/pprof/pprof.Zen.alloc_objects.alloc_space.inuse_objects.inuse_space.002.pb.gz
File: Zen
Type: inuse_space
Time: 2025-06-27 02:32:18 +05
Entering interactive mode (type "help" for commands, "o" for options)
(pprof) pdf
Generating report in profile001.pdf
(pprof) svg
Generating report in profile001.svg
@anfragment I added a debug.SetGCPercent(25) statement to the main function and I am seeing 370 MB usage on Windows 11 Pro now
@ilovelinabell probably not something we can run in production though, right?
I added it to the code so it will run in the main function. I will play around with the value and tell you if we should merge in my changes
@anfragment Here is a new PR to shrink mem usage a bit more https://github.com/ZenPrivacy/zen-desktop/pull/388
Took a quick look in https://github.com/ZenPrivacy/zen-desktop/pull/458 maybe it will help, can someone try it out?
