Segfault when inotify instance limit is reached

[BrenBarn]

• [x] I HAVE READ AND ACKNOWLEDGED THIS HOWTO ARTICLE:
  Submitting a MiKTeX problem report

I spent some time trying to debug a previous issue I reported. Various Miktex commands (such as miktex-console and initexmf) were segfaulting immediately after installing the package on Kubuntu 24.04. I was able to prevent the segfault by increasing the inotify user instance limit.

Although I initially was focusing on miktex-console I decided to try to debug initexmf to rule out Qt issues. I found that the segfault occurred in SessionImpl::Close. I spent some time going down rabbit holes because it worked on a different user account so I thought it had something to do with user-specific config files. However, I eventually found that the same SessionImpl::Close was reached on a successful run (on a different user account), after the output was printed, suggesting that the error wasn't really in that code but occurred earlier and the segfault happened during the shutdown/cleanup process.

I used catch throw to see that an exception was thrown in Session::FatalCrtError in a call stack that included unxFileSystemWatcher::unxFileSystemWatcher. Seeing the inotify call there led me to check if it could be an inotify error and I eventually raised the limit in /proc/sys/fs/inotify/max_user_instances and that prevented the segfault. (This also prevents the segfault in miktex-console.)

I don't know enough about Miktex internals to know why the inotify failure leads to a segfault later in SessionImpl::Close. I'm creating this issue to give more information about what caused the crash. It still seems to me that there is likely a bug, because Miktex should fail gracefully if the inotify limit is reached (e.g., with a message like "try increasing the inotify instance limit"), rather than segfaulting with no informative error message.

[github-actions[bot]]

This issue has been marked as stale due to inactivity.

[BrenBarn]

It's stale because the issue was not acknowledged. The issue still exists.