napari-nD-annotator Permission denied errors in `self.clean

Describe the bug A collaborator of mine is receiving a permission denied error simply when opening the Annotator plugin. She is working on a workstation where she is having limited permissions (i.e., no sudo access).

This is the full traceback of the error message:

Traceback (most recent call last):
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/menus/plugins_menu.py", line 105, in _add_toggle_widget
    self._win.add_plugin_dock_widget(*key)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/qt_main_window.py", line 897, in add_plugin_dock_widget
    wdg = _instantiate_dock_widget(
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/qt_main_window.py", line 1551, in _instantiate_dock_widget
    return wdg_cls(**kwargs)
           ^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/_widgets/annotator_module.py", line 70, in __init__
    self.minimal_contour_widget = MinimalContourWidget(viewer, self)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/_widgets/minimal_contour_widget.py", line 102, in __init__
    self.feature_manager = FeatureManager(viewer)
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/minimal_contour/feature_manager.py", line 24, in __init__
    self.clean_tmp()
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/minimal_contour/feature_manager.py", line 136, in clean_tmp
    shutil.rmtree(fold)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 759, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 674, in _rmtree_safe_fd
    topfd = os.open(name, os.O_RDONLY | os.O_NONBLOCK, dir_fd=dirfd)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/tmp/tmpl6dib9_u_nd_annotator'
Traceback (most recent call last):
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/menus/plugins_menu.py", line 105, in _add_toggle_widget
    self._win.add_plugin_dock_widget(*key)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/qt_main_window.py", line 897, in add_plugin_dock_widget
    wdg = _instantiate_dock_widget(
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/qt_main_window.py", line 1551, in _instantiate_dock_widget
    return wdg_cls(**kwargs)
           ^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/_widgets/annotator_module.py", line 70, in __init__
    self.minimal_contour_widget = MinimalContourWidget(viewer, self)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/_widgets/minimal_contour_widget.py", line 102, in __init__
    self.feature_manager = FeatureManager(viewer)
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/minimal_contour/feature_manager.py", line 24, in __init__
    self.clean_tmp()
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/minimal_contour/feature_manager.py", line 136, in clean_tmp
    shutil.rmtree(fold)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 759, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 674, in _rmtree_safe_fd
    topfd = os.open(name, os.O_RDONLY | os.O_NONBLOCK, dir_fd=dirfd)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/tmp/tmpl6dib9_u_nd_annotator'
Traceback (most recent call last):
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/menus/plugins_menu.py", line 105, in _add_toggle_widget
    self._win.add_plugin_dock_widget(*key)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/qt_main_window.py", line 897, in add_plugin_dock_widget
    wdg = _instantiate_dock_widget(
          ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari/_qt/qt_main_window.py", line 1551, in _instantiate_dock_widget
    return wdg_cls(**kwargs)
           ^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/_widgets/annotator_module.py", line 70, in __init__
    self.minimal_contour_widget = MinimalContourWidget(viewer, self)
                                  ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/_widgets/minimal_contour_widget.py", line 102, in __init__
    self.feature_manager = FeatureManager(viewer)
                           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/minimal_contour/feature_manager.py", line 24, in __init__
    self.clean_tmp()
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/site-packages/napari_nd_annotator/minimal_contour/feature_manager.py", line 136, in clean_tmp
    shutil.rmtree(fold)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 759, in rmtree
    _rmtree_safe_fd(stack, onexc)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 703, in _rmtree_safe_fd
    onexc(func, path, err)
  File "/home/pol_dye/saja957d/miniconda3/envs/gt_env/lib/python3.12/shutil.py", line 674, in _rmtree_safe_fd
    topfd = os.open(name, os.O_RDONLY | os.O_NONBLOCK, dir_fd=dirfd)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
PermissionError: [Errno 13] Permission denied: '/tmp/tmpl6dib9_u_nd_annotator'

It seems like the tempfile package is trying to write something into the root tmpfile rather than into the users workspace, which would be a better go-to.

To Reproduce Steps to reproduce the behavior:

Open plugin on Linux system as non-sudo user

Expected behavior Plugin widget should open without error

napari info Copy information from Help -> napari Info.

Suggested fix

Something to set the /tmp file to the user's homedir should probably do the trick, e.g.:

os.environ['TMPDIR'] = os.path.expanduser('~/tmp')

Jun 28 '24 12:06 jo-mueller

Hey @jo-mueller, thanks for the report! I will check the problem asap. But this bug also shows, that the plugin needs better exception handling, as problems with persistence shouldn't break the whole plugin... By the way, recently I realized that the magicgui package can store widget values on disk, so I'll check whether it would be a feasible solution. Thanks again!

Jul 01 '24 11:07 bauerdavid

Hi @bauerdavid ,

thanks for the superfast reply :) Maybe another way to avoid this would be to write the plugin as a Qt widget from the get-go, rather than going the magicgui route? It's probably something for the next major version, but it would solve these persistence error problems and would maybe also make some of the "overhead layers" in the viewer unnecessary? (I take it that these are used to pass data between instances of the annotator plugin?)

Things like the Qt designer would certainly make at least the design of the UI relatively easy on that part.

Edit: Besides the design considerations, simply commenting out the self.clean_tmp() made the plugin functional again. But I don't know the implications for ither functionality...

Jul 01 '24 11:07 jo-mueller

Okay, I was completely wrong :D This is not about widget persistence, but the temporarily stored image features used by minimal contour. My guess is that the temp files are not closed properly, so they cannot be removed.

Jul 02 '24 08:07 bauerdavid

The temp files are supposed to be removed by clean() when Python exits (achieved by atexit). The self.clean_tmp function runs on startup to remove any leftover temp files in case of something going wrong with the cleanup. This was done to make sure that the disk will never be flooded with leftover, as on Windows I've seen some cases where napari crashed and these files weren't removed.

I've looked into the code of the clean() method, and I see the two empty except blocks with no Exception type provided, which can be very dangerous.

So my theory is:

when clean() runs, there's an exception, but is not visible due to the empty except block
whatever prevents clean() from removing the temp files, also prevents clean_tmp() from doing so

Jul 02 '24 14:07 bauerdavid

I created a new branch and changed the handling of exceptions (from completely ignoring to printing the stack trace). Could you try it on the problematic machine? What I hope would happen is 1) the plugin won't break on exception and 2) We'll see what the problem is. Unfortunately there's some C/C++ and Cython code in the repository, if you have issues with installing, please let me know.

Jul 03 '24 09:07 bauerdavid

Hi @bauerdavid ,

sorry for not replying in such a long time. I just tried from your branch and I still receive an error but the traceback does seem to be a bit more verbose :)

Full traceback:

---------------------------------------------------------------------------
PermissionError                           Traceback (most recent call last)
File ~/miniforge3/envs/some_env/lib/python3.9/site-packages/napari/_qt/menus/plugins_menu.py:105, in PluginsMenu._add_plugin_actions.<locals>._add_toggle_widget(key=('napari-nD-annotator', 'Annotation Toolbox'), hook_type='dock')
    102     return
    104 if hook_type == 'dock':
--> 105     self._win.add_plugin_dock_widget(*key)
        key = ('napari-nD-annotator', 'Annotation Toolbox')
        self._win = <napari._qt.qt_main_window.Window object at 0x7f9a638a3190>
        self = <napari._qt.menus.plugins_menu.PluginsMenu object at 0x7f993c7b6af0>
    106 else:
    107     self._win._add_plugin_function_widget(*key)

File ~/miniforge3/envs/some_env/lib/python3.9/site-packages/napari/_qt/qt_main_window.py:897, in Window.add_plugin_dock_widget(self=<napari._qt.qt_main_window.Window object>, plugin_name='napari-nD-annotator', widget_name='Annotation Toolbox', tabify=False)
    894         wdg = wdg._magic_widget
    895     return dock_widget, wdg
--> 897 wdg = _instantiate_dock_widget(
        Widget = <class 'napari_nd_annotator._widgets.annotator_module.AnnotatorWidget'>
        self = <napari._qt.qt_main_window.Window object at 0x7f9a638a3190>
    898     Widget, cast('Viewer', self._qt_viewer.viewer)
    899 )
    901 # Add dock widget
    902 dock_kwargs.pop('name', None)

File ~/miniforge3/envs/some_env/lib/python3.9/site-packages/napari/_qt/qt_main_window.py:1551, in _instantiate_dock_widget(wdg_cls=<class 'napari_nd_annotator._widgets.annotator_module.AnnotatorWidget'>, viewer=Viewer(camera=Camera(center=(0.0, 127.5, 127.5),...ouse_drag_gen={}, _mouse_wheel_gen={}, keymap={}))
   1546             break
   1547         # cannot look for param.kind == param.VAR_KEYWORD because
   1548         # QWidget allows **kwargs but errs on unknown keyword arguments
   1549 
   1550 # instantiate the widget
-> 1551 return wdg_cls(**kwargs)
        kwargs = {'viewer': Viewer(camera=Camera(center=(0.0, 127.5, 127.5), zoom=4.787109375, angles=(0.0, 0.0, 90.0), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(29.0, 204.63067139229273, -244.5655622934073), scaled=True, size=10, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=3, ndisplay=2, last_used=0, range=((0.0, 60.0, 1.0), (0.0, 256.0, 1.0), (0.0, 256.0, 1.0)), current_step=(29, 127, 127), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'membrane' at 0x7f98e87e5f70>, <Image layer 'nuclei' at 0x7f98e838ef70>, <Labels layer 'Labels' at 0x7f98c869bc70>], help='use <1> for activate the label eraser, use <2> for activate the paint brush, use <3> for activate the fill bucket, use <4> for pick mode', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x7f9a85705af0>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={})}
        wdg_cls = <class 'napari_nd_annotator._widgets.annotator_module.AnnotatorWidget'>

File ~/Github/napari-nD-annotator/src/napari_nd_annotator/_widgets/annotator_module.py:70, in AnnotatorWidget.__init__(self=<napari_nd_annotator._widgets.annotator_module.AnnotatorWidget object>, viewer=Viewer(camera=Camera(center=(0.0, 127.5, 127.5),...ouse_drag_gen={}, _mouse_wheel_gen={}, keymap={}))
     67 self.interpolation_widget = InterpolationWidget(viewer, self)
     68 tabs_widget.addTab(self.interpolation_widget, "Interpolation")
---> 70 self.minimal_contour_widget = MinimalContourWidget(viewer, self)
        viewer = Viewer(camera=Camera(center=(0.0, 127.5, 127.5), zoom=4.787109375, angles=(0.0, 0.0, 90.0), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(29.0, 204.63067139229273, -244.5655622934073), scaled=True, size=10, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=3, ndisplay=2, last_used=0, range=((0.0, 60.0, 1.0), (0.0, 256.0, 1.0), (0.0, 256.0, 1.0)), current_step=(29, 127, 127), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'membrane' at 0x7f98e87e5f70>, <Image layer 'nuclei' at 0x7f98e838ef70>, <Labels layer 'Labels' at 0x7f98c869bc70>], help='use <1> for activate the label eraser, use <2> for activate the paint brush, use <3> for activate the fill bucket, use <4> for pick mode', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x7f9a85705af0>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={})
        self = <napari_nd_annotator._widgets.annotator_module.AnnotatorWidget object at 0x7f9a634951f0>
        MinimalContourWidget = <class 'napari_nd_annotator._widgets.minimal_contour_widget.MinimalContourWidget'>
     71 tabs_widget.addTab(self.minimal_contour_widget, "Minimal Contour")
     73 if MinimalSurfaceWidget is not None:

File ~/Github/napari-nD-annotator/src/napari_nd_annotator/_widgets/minimal_contour_widget.py:102, in MinimalContourWidget.__init__(self=<napari_nd_annotator._widgets.minimal_contour_widget.MinimalContourWidget object>, viewer=Viewer(camera=Camera(center=(0.0, 127.5, 127.5),...ouse_drag_gen={}, _mouse_wheel_gen={}, keymap={}), parent=<napari_nd_annotator._widgets.annotator_module.AnnotatorWidget object>)
    100 self.feature_inverted = False
    101 self.prev_timer_id = None
--> 102 self.feature_manager = FeatureManager(viewer)
        viewer = Viewer(camera=Camera(center=(0.0, 127.5, 127.5), zoom=4.787109375, angles=(0.0, 0.0, 90.0), perspective=0.0, mouse_pan=True, mouse_zoom=True), cursor=Cursor(position=(29.0, 204.63067139229273, -244.5655622934073), scaled=True, size=10, style=<CursorStyle.STANDARD: 'standard'>), dims=Dims(ndim=3, ndisplay=2, last_used=0, range=((0.0, 60.0, 1.0), (0.0, 256.0, 1.0), (0.0, 256.0, 1.0)), current_step=(29, 127, 127), order=(0, 1, 2), axis_labels=('0', '1', '2')), grid=GridCanvas(stride=1, shape=(-1, -1), enabled=False), layers=[<Image layer 'membrane' at 0x7f98e87e5f70>, <Image layer 'nuclei' at 0x7f98e838ef70>, <Labels layer 'Labels' at 0x7f98c869bc70>], help='use <1> for activate the label eraser, use <2> for activate the paint brush, use <3> for activate the fill bucket, use <4> for pick mode', status='', tooltip=Tooltip(visible=False, text=''), theme='dark', title='napari', mouse_over_canvas=False, mouse_move_callbacks=[], mouse_drag_callbacks=[], mouse_double_click_callbacks=[], mouse_wheel_callbacks=[<function dims_scroll at 0x7f9a85705af0>], _persisted_mouse_event={}, _mouse_drag_gen={}, _mouse_wheel_gen={}, keymap={})
        self = <napari_nd_annotator._widgets.minimal_contour_widget.MinimalContourWidget object at 0x7f980c05f550>
    104 # Ctrl+Z handling
    105 self.change_idx = dict()

File ~/Github/napari-nD-annotator/src/napari_nd_annotator/minimal_contour/feature_manager.py:26, in FeatureManager.__init__(self=<napari_nd_annotator.minimal_contour.feature_manager.FeatureManager object>, viewer=Viewer(camera=Camera(center=(0.0, 127.5, 127.5),...ouse_drag_gen={}, _mouse_wheel_gen={}, keymap={}))
     24 self.memmaps: list[Optional[Union[np.ndarray, np.memmap]]] = [None, None]
     25 self.slices_calculated = dict()
---> 26 self.clean_tmp()
        self = <napari_nd_annotator.minimal_contour.feature_manager.FeatureManager object at 0x7f980c02a7f0>
     27 self.temp_folder = tempfile.mkdtemp(suffix=TEMP_SUFFIX)
     28 # map layers to file prefix

File ~/Github/napari-nD-annotator/src/napari_nd_annotator/minimal_contour/feature_manager.py:133, in FeatureManager.clean_tmp(self=<napari_nd_annotator.minimal_contour.feature_manager.FeatureManager object>)
    131 temp_folders = glob.glob(os.path.join(temp_dir, "%s*%s" % (tempfile.gettempprefix(), TEMP_SUFFIX)))
    132 for fold in temp_folders:
--> 133     shutil.rmtree(fold)
        fold = '/tmp/tmpolmrems8_nd_annotator'

File ~/miniforge3/envs/some_env/lib/python3.9/shutil.py:730, in rmtree(path='/tmp/tmpolmrems8_nd_annotator', ignore_errors=False, onerror=<function rmtree.<locals>.onerror>)
    728     fd_closed = False
    729 except Exception:
--> 730     onerror(os.open, path, sys.exc_info())
        path = '/tmp/tmpolmrems8_nd_annotator'
    731     return
    732 try:

File ~/miniforge3/envs/some_env/lib/python3.9/shutil.py:727, in rmtree(path='/tmp/tmpolmrems8_nd_annotator', ignore_errors=False, onerror=<function rmtree.<locals>.onerror>)
    725     return
    726 try:
--> 727     fd = os.open(path, os.O_RDONLY)
        path = '/tmp/tmpolmrems8_nd_annotator'
        os.O_RDONLY = 0
    728     fd_closed = False
    729 except Exception:

PermissionError: [Errno 13] Permission denied: '/tmp/tmpolmrems8_nd_annotator'

Jul 23 '24 13:07 jo-mueller

What puzzles me is that the script had permission to create the folder, but afterwards it doesn't have permission to delete it. Could you check if the user has write access, and also whether you can delete the folder/its contents manually? I started to experiment a bit, and what I found is that I'm actually able to remove the files the memmaps are using manually even when the memmaps are still in use. This means that removing currently accessed files is not the problem. I'm a bit stuck on what is causing this, I cannot reproduce it on my machine...

Jul 26 '24 08:07 bauerdavid

I just tried to forcefully remove it from the command line and got this:

(pytorch) johamuel@biapol-ws1:/tmp$ rm -rf tmp9zlwxwjq_nd_annotator/
rm: cannot remove 'tmp9zlwxwjq_nd_annotator/': Operation not permitted

I had a look at the permissions of these temporary files and this is more enlightening, I think:

drwx------  2 saja957d pol_dye      4096 Jul  8 16:06 tmp9zlwxwjq_nd_annotator
drwx------  2 saja957d pol_dye      4096 Jul  8 13:56 tmpolmrems8_nd_annotator

There are somehow two temporary folders from the nd annotator that were created by another user. Is it possible that the nd-annotator tries to reuse these and then fails in removing them?

Jul 26 '24 10:07 jo-mueller

yup, this should be it. Added code for ignoring PermissionErrors, check if it works.

Jul 26 '24 11:07 bauerdavid

Checked it, the change does fix the problem.

Aug 09 '24 12:08 bauerdavid

Permission denied errors in `self.clean_tmp()`