dua-cli icon indicating copy to clipboard operation
dua-cli copied to clipboard

Can cause memory leak for xdg-desktop-portal

Open tim77 opened this issue 5 years ago • 6 comments

Memory leak happens when running dua on root / partition, see related bug.

How to reproduce:

Just install some flatpak apps system-wide and run them before. Then run dua on /.

tim77 avatar Jun 28 '20 06:06 tim77

By the looks of it, the proc filesystem contains a cycle, trapping unsuspecting programs in it. If that's truly the case, I wonder if that's a bug in Flatpack more than a bug in any tool that rightfully (?) expects filesystems trees to be acyclic.

Regarding memory usage: In order to handle hard-links, a mapping is maintained from inodes with more than one reference. In interactive mode, all traversed file paths will be kept in memory for later traversal. That's not a leak, but the cost of providing certain capabilities.

If the proc filesystem somehow does produce cycles, this issue would probably become something like handle cyclic filesystems gracefully , as right now it would probably go into an infinite loop.

In order to make this issue actionable, more information is required, maybe you can help answering these questions of mine:

  • Can this issue be reproduced with dua v2.6.1?
  • Does the proc file system contain cycles?
  • Does the issue also occour when running dua /proc?
  • How does xdg-document-portal relate with dua?

Thank you

Byron avatar Jun 28 '20 06:06 Byron

  • Can this issue be reproduced with dua v2.6.1? Yes, using v2.6.1.
  • Does the proc file system contain cycles? :soon:
  • Does the issue also occour when running dua /proc? I could say there is no issue if i try dua /proc
  • How does xdg-document-portal relate with dua? xdg-document-portal not related with dua but because of popularity users may got the same serious problem IMO and maybe good idea to workaround this somehow for now and exclude this problematic path from dua by default. For example baobab doesn't have this issue but baobab is view disk space usage utility as well.

tim77 avatar Jun 28 '20 06:06 tim77

  • Does the proc file system contain cycles?

Seems like not. Problem comes from here dua /run/user/

tim77 avatar Jun 28 '20 11:06 tim77

Thanks for looking into this and for narrowing down the cause of the issue. As I am on MacOS, I can't exactly reproduce this issue easily, and would be relying on a PR fixing it or on way more information on what you think is causing this within /run/user. Thanks again

Byron avatar Jun 28 '20 11:06 Byron

If be more precisely: dua /run/user/1000/doc/ causing this. There is place for flatpak portals but what exactly causing this i am still trying to investigate... Maybe we can safely exclude this path for now in dua?

tim77 avatar Jun 30 '20 13:06 tim77

Thanks for digging in! I believe that hardcoding exceptions without understanding the root cause is not really going to help in the long-term, and thus would be happy to wait until you figured out what is causing this.

By the way: if you run dua i /run/user you can interactively dig into the directory as it is parsing it - maybe that helps to see what's going on.

Byron avatar Jul 01 '20 06:07 Byron