Folders plugin always fails on special directories

[Ellpeck]

Is your feature request related to a problem? Please describe. Hi again!

In addition/relation to #2494, I found another issue: The folders plugin errors when it fails to detect the size of the proc directory on Linux, which causes an issue for displaying the size of directories like the docker volume directory, since those also contain versions of proc.

Obviously, this makes sense and is in line with other tools, but those tools just display an error and continue scanning directory sizes instead of failing altogether. This is ncdu for example:

Warning: error scanning /proc/579310/task/579311/fd/23
    some directory sizes may not be correct

Describe the solution you'd like Potentially, it would also make sense for the folders plugin to either display an error and continue scanning rather than failing outright, or to be able to specify a set of file names (maybe through a regular expression or a glob pattern?) that should be ignored when scanning, or to ignore proc specifically since it's a special directory.

Additional context As I said at the top, this issue is somewhat in relation to #2494, which was fixed incredibly quickly & effectively, so thanks so much again for that!!

[nicolargo]

Hi @Ellpeck

Please run Glances in debug mode (-d) and search for the "Cannot get folder size" pattern in the Glances log file and copy paste the result.

On my side, with the following configuration:

folder_1_path=/proc

I have:

2023-08-06 12:59:43,072 -- ERROR -- Cannot get folder size (/proc). Error: [Errno 13] Permission denied: '/proc/fs/nfsd/clients'

[Ellpeck]

With my configuration (which, again, I'd like to use to display the size of the docker volumes and related files)

folder_4_path=/var/lib/docker
folder_4_refresh=3600

I get pretty much the same message:

2023-07-17 07:59:56,509 -- DEBUG -- Cannot get folder size (/var/lib/docker). Error: [Errno 13] Permission denied: '/var/lib/docker/overlay2/8143ce91a8591b8c3cfd9cd980b024cf14cac212a504c075b5ebc43a9ff0189d/merged/proc/1/task/1/fdinfo'

[nicolargo]

Done in the develop branch (will be release in Glances 4.0.0).

The new behavor is the following: if a file (or subfolder) is not readable, it is ignored by Glances and a special char (?) / color is displayed in the console and Web UI.

@Ellpeck Can you test it ?

[Ellpeck]

Hi! I tested the change as you requested. Unfortunately, using the dev branch release from 2 days ago (https://hub.docker.com/layers/nicolargo/glances/dev/images/sha256-cb8889ab67b41aa535b2ac7b66edf15be62d11e046c919c27833ed39f6508aca), using a directory that causes permission issues causes Glances to fail to start up entirely. The full stack trace is the following:

glances    | Traceback (most recent call last):
glances    |   File "<frozen runpy>", line 198, in _run_module_as_main
glances    |   File "<frozen runpy>", line 88, in _run_code
glances    |   File "/app/glances/__main__.py", line 18, in <module>
glances    |     glances.main()
glances    |   File "/app/glances/__init__.py", line 183, in main
glances    |     start(config=core.get_config(), args=core.get_args())
glances    |   File "/app/glances/__init__.py", line 106, in start
glances    |     mode = GlancesMode(config=config, args=args)
glances    |            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
glances    |   File "/app/glances/webserver.py", line 31, in __init__
glances    |     self.stats.update()
glances    |   File "/app/glances/stats.py", line 223, in update
glances    |     self._plugins[p].update()
glances    |   File "/app/glances/plugins/plugin/model.py", line 1113, in wrapper
glances    |     ret = fct(self, *args, **kw)
glances    |           ^^^^^^^^^^^^^^^^^^^^^^
glances    |   File "/app/glances/plugins/plugin/model.py", line 1130, in wrapper
glances    |     ret = fct(*args, **kw)
glances    |           ^^^^^^^^^^^^^^^^
glances    |   File "/app/glances/plugins/folders/model.py", line 53, in update
glances    |     self.glances_folders.update(key=self.get_key())
glances    |   File "/app/glances/folder_list.py", line 133, in update
glances    |     self.__folder_list[i]['size'], self.__folder_list[i]['errno'] = folder_size(self.path(i))
glances    |                                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^
glances    |   File "/app/glances/globals.py", line 392, in folder_size
glances    |     ret = folder_size(os.path.join(path, f.name), ret_err)
glances    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
glances    |   File "/app/glances/globals.py", line 392, in folder_size
glances    |     ret = folder_size(os.path.join(path, f.name), ret_err)
glances    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
glances    |   File "/app/glances/globals.py", line 392, in folder_size
glances    |     ret = folder_size(os.path.join(path, f.name), ret_err)
glances    |           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
glances    |   [Previous line repeated 3 more times]
glances    |   File "/app/glances/globals.py", line 390, in folder_size
glances    |     for f in f_list:
glances    | PermissionError: [Errno 13] Permission denied: '/fs/var/lib/docker/overlay2/5713828a7a89c1d71108ac5f1316cf9b856bce8975367eed806a8ab72b8d8ac9/merged/proc/1/map_files'

[nicolargo]

Patch merged into develop for the PermissionError.

Will be release in Glances 4.0.0.

Thanks for the report @Ellpeck (and sorry for the delay).