aiida-core
aiida-core copied to clipboard
🐛 `verdi storage maintain`: `psutil.AccessDenied` when running `--full`
Describe the bug
❯ verdi storage maintain --full
Warning:
In order to safely perform the full maintenance operations on the internal storage, the profile dev needs to be locked. This means that no other process will be able to access it and will fail instead. Moreover, if any process is already using the profile, the locking attempt will fail and you will have to either look for these processes and kill them or wait for them to stop by themselves. Note that this includes verdi shells, daemon workers, scripts that manually load it, etc.
For performing maintenance operations that are safe to run while actively using AiiDA, just run `verdi storage maintain` without the `--full` flag.
Are you sure you want continue in this mode? [y/N]: y
Traceback (most recent call last):
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/psutil/_psosx.py", line 346, in wrapper
return fun(self, *args, **kwargs)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/psutil/_psosx.py", line 404, in cmdline
return cext.proc_cmdline(self.pid)
PermissionError: [Errno 1] Operation not permitted (originated from sysctl(KERN_PROCARGS2))
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/mbercx/.aiida_venvs/super/bin/verdi", line 8, in <module>
sys.exit(verdi())
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/core.py", line 1157, in __call__
return self.main(*args, **kwargs)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/core.py", line 1078, in main
rv = self.invoke(ctx)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/core.py", line 1688, in invoke
return _process_result(sub_ctx.command.invoke(sub_ctx))
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/core.py", line 1434, in invoke
return ctx.invoke(self.callback, **ctx.params)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/core.py", line 783, in invoke
return __callback(*args, **kwargs)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/aiida/cmdline/utils/decorators.py", line 73, in wrapper
return wrapped(*args, **kwargs)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/click/decorators.py", line 33, in new_func
return f(get_current_context(), *args, **kwargs)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/aiida/cmdline/commands/cmd_storage.py", line 165, in storage_maintain
storage.maintain(full=full, dry_run=dry_run, compress=compress)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/aiida/storage/psql_dos/backend.py", line 384, in maintain
with maintenance_context():
File "/opt/homebrew/Cellar/[email protected]/3.10.13_1/Frameworks/Python.framework/Versions/3.10/lib/python3.10/contextlib.py", line 135, in __enter__
return next(self.gen)
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/aiida/manage/profile_access.py", line 112, in lock
self._clear_stale_pid_files()
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/aiida/manage/profile_access.py", line 175, in _clear_stale_pid_files
if path.read_text() != str(process.cmdline()):
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/psutil/__init__.py", line 686, in cmdline
return self._proc.cmdline()
File "/Users/mbercx/.aiida_venvs/super/lib/python3.10/site-packages/psutil/_psosx.py", line 353, in wrapper
raise AccessDenied(self.pid, self._name)
psutil.AccessDenied: (pid=756)
The process 756
seems to be something related to a security service of macOS:
❯ ps -p 756
PID TTY TIME CMD
756 ?? 0:00.04 /System/Library/PrivateFrameworks/XprotectFramework.framework/Versions/A/XPCServices/XProtectBehaviorService.xpc/Contents/MacOS/XProtectBehaviorService
I was curious to see if other profiles in other environments faced a similar issue. There the error was the same, but the PID was different:
❯ ps -p 385
PID TTY TIME CMD
385 ?? 0:00.04 /usr/libexec/containermanagerd --runmode=agent --user-container-mode=current --bundle-container-mode=proxy --system-container-mode=none
Steps to reproduce
I'm not sure if it's easy to reproduce. It is likely to be macOS-specific. At least I can reproduce it plenty ^^
Your environment
- Operating system [e.g. Linux]: macOS Ventura 13.4 (22F2063)
- Python version [e.g. 3.7.1]: Python 3.10.13
- aiida-core version [e.g. 1.2.1]: 2.4.1