maestral icon indicating copy to clipboard operation
maestral copied to clipboard
verified publisher icon samschott

Maestral becomes unresponsive while indexing

Open madrobby opened this issue 4 years ago • 3 comments

Describe the bug Maestral becomes unresponsive while indexing. Trying to click the menubar icon just shows the SBOD cursor. The app is running (I can see sync INFO log entries) but the dock icon says the app isn't running.

Killing the app via CLI and restarting it results in the same behavior, the daemon is happily working but the menubar icon is light grey and can't be interacted with (SBOD).

This is the initial sync, with selected folders. Our overall Dropbox has 10TB+ but I'm only trying to sync a few hundred GB.

To Reproduce Unknown

Expected behaviour App responds to user

System:

  • Maestral version: 1.5.1
  • Python version: 5.15.4
  • OS: macOS Monterey 12.0.1 on Intel

Additional context Screenshots and log:

2021-10-29 07:24:57 sync INFO: Syncing ↓ 9/251
2021-10-29 07:25:00 sync INFO: Indexing 23930...
2021-10-29 07:25:02 sync INFO: Indexing 24430...
2021-10-29 07:25:04 sync INFO: Indexing 24930...
2021-10-29 07:25:06 sync INFO: Indexing 25430...
2021-10-29 07:25:09 sync INFO: Indexing 25930...
2021-10-29 07:25:11 sync INFO: Indexing 26430...
2021-10-29 07:25:13 sync INFO: Indexing 26930...
2021-10-29 07:25:15 sync INFO: Indexing 27430...
2021-10-29 07:25:16 sync INFO: Indexing 27930...
2021-10-29 07:25:17 sync INFO: Indexing 28430...
2021-10-29 07:25:19 sync INFO: Indexing 28930...

Screen Shot 2021-10-29 at 7 34 15 AM Screen Shot 2021-10-29 at 7 34 22 AM

$ ps aux | grep "Maestral"
thomasfuchs      34925  75.9 27.5 65366036 4605940   ??  R     7:25AM   7:35.49 /Applications/Maestral.app/Contents/MacOS/Maestral
thomasfuchs      34719   3.8  0.5 34449084  84388   ??  SN    7:13AM   3:44.75 /Applications/Maestral.app/Contents/MacOS/Maestral -c import maestral.daemon; maestral.daemon.start_maestral_daemon("maestral")

madrobby avatar Oct 29 '21 14:10 madrobby

Oh, that is funky. Unfortunately, the way indexing is currently set up requires it to index the entire 10 TB + of files. It will however skip any excluded files or folders during the download phase. With the current Dropbox APIs, it is much easier to handle indexing this way rather than skip excluded folders completely.

  1. Does the GUI become unresponsive straight away or only after some time? In the second case, is that point reproducible?
  2. If you kill and restart Maestral, does it start of indexing at the beginning or resume from where it left off?
  3. What do CLI commands such as maestral status or maestral activity return? Or do they also hang? If they also hang, the issue is likely with the daemon rather than the GUI.

Since the log is still being populated, the daemon is clearly doing something, but all indexing should be carried out in threads while the API to the CLI and GUI remains responsive...

samschott avatar Oct 29 '21 17:10 samschott

  1. GUI becomes unresponsive immediately. This is reproducible, I can just kill the GUI app and restart it and it happens again.
  2. I don't know because I can't see what's going on. Clicking the icon just shows a SBOD.
  3. They also hang apparently... I do get some logs and activity stuff but it's strewn with errors. I also has a full-on crash. Attaching logs.
$ maestral activity
Traceback (most recent call last):
  File "Contents/Resources/Support/Python/Resources/lib/python3.9/runpy.py", line 197, in _run_module_as_main
  File "Contents/Resources/Support/Python/Resources/lib/python3.9/runpy.py", line 87, in _run_code
  File "Contents/Resources/app/maestral_cocoa/__main__.py", line 41, in <module>
  File "Contents/Resources/app/maestral_cocoa/__main__.py", line 36, in freeze_support_cli
  File "Contents/Resources/app_packages/click/core.py", line 1128, in __call__
  File "Contents/Resources/app_packages/click/core.py", line 1053, in main
  File "Contents/Resources/app_packages/click/core.py", line 1659, in invoke
  File "Contents/Resources/app_packages/click/core.py", line 1395, in invoke
  File "Contents/Resources/app_packages/click/core.py", line 754, in invoke
  File "Contents/Resources/app_packages/maestral/cli.py", line 228, in wrapper
  File "Contents/Resources/app_packages/maestral/cli.py", line 1084, in activity
  File "Contents/Resources/Support/Python/Resources/lib/python3.9/curses/__init__.py", line 94, in wrapper
  File "Contents/Resources/app_packages/maestral/cli.py", line 1029, in curses_loop
  File "Contents/Resources/app_packages/maestral/daemon.py", line 676, in __getattr__
  File "Contents/Resources/app_packages/Pyro5/client.py", line 94, in __getattr__
  File "Contents/Resources/app_packages/Pyro5/client.py", line 243, in _pyroInvoke
RuntimeError: Set changed size during iteration
 +--- This exception occured remotely (Pyro) - Remote traceback:
 | Traceback (most recent call last):
 |   File "Contents/Resources/app_packages/Pyro5/server.py", line 453, in handleRequest
 |   File "Contents/Resources/app_packages/Pyro5/server.py", line 946, in _get_exposed_property_value
 |   File "Contents/Resources/app_packages/maestral/main.py", line 485, in sync_errors
 |   File "Contents/Resources/app_packages/maestral/main.py", line 485, in <listcomp>
 | RuntimeError: Set changed size during iteration
 +--- End of remote traceback

Screen Shot 2021-10-31 at 10 33 11 AM

Example log errors:

OSError: [Errno 5] Input/output error: '/Users/thomasfuchs/Dropbox (Slash7)/Amy/Photos/Backup iPhone 3GS/IMG_1452.JPG'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "Contents/Resources/app_packages/maestral/sync.py", line 2209, in _create_remote_entry
  File "Contents/Resources/app_packages/maestral/sync.py", line 2439, in _on_local_created
  File "Contents/Resources/app_packages/maestral/client.py", line 743, in upload
  File "Contents/Resources/Support/Python/Resources/lib/python3.9/contextlib.py", line 135, in __exit__
  File "Contents/Resources/app_packages/maestral/client.py", line 143, in convert_api_errors
maestral.errors.FileReadError: Could not sync file or folder. Could not access file. Errno 5: Input/output error.
2021-10-31 10:36:54 sync INFO: Could not sync IMG_1308.JPG
Traceback (most recent call last):
  File "Contents/Resources/app_packages/maestral/client.py", line 126, in convert_api_errors
  File "Contents/Resources/app_packages/maestral/client.py", line 674, in upload
OSError: [Errno 5] Input/output error: '/Users/thomasfuchs/Dropbox (Slash7)/Amy/Photos/Backup iPhone 3GS/IMG_1308.JPG'

Date/Time:        2021-10-31 10:30:02.020 -0700
End time:         2021-10-31 10:32:01.135 -0700
OS Version:       macOS 12.0.1 (Build 21A559)
Architecture:     x86_64h
Report Version:   35
Incident Identifier: C5D7B59B-2784-4179-9815-26083D4CC834

Data Source:      Microstackshots
Shared Cache:     3D05845F-3F65-358F-9EBF-2236E772AC01 slid base address 0x7ff810f8c000, slide 0x10f8c000

Command:          Maestral
Path:             /Applications/Maestral.app/Contents/MacOS/Maestral
Identifier:       com.samschott.maestral
Version:          1.5.1 (40)
Team ID:          G34LNR8C4Y
Architecture:     x86_64
Parent:           UNKNOWN [1]
PID:              52956

Event:            cpu usage
Action taken:     none
CPU:              90 seconds cpu time over 119 seconds (76% cpu average), exceeding limit of 50% cpu over 180 seconds
CPU limit:        90s
Limit duration:   180s
CPU used:         90s
CPU duration:     119s
Duration:         119.12s
Duration Sampled: 117.31s
Steps:            33

Hardware model:   MacBookPro13,2
Active cpus:      4
HW page size:     4096
VM page size:     4096

Fan speed:        2354 rpm
Advisory levels:  Battery -> 3, User -> 2, ThermalPressure -> 1, Combined -> 2
Free disk space:  101.42 GB/931.42 GB, low space threshold 3072 MB

Heaviest stack for the target process:
  30  ??? [0x7ff7bb3940f0]
  30  ffi_call_unix64 + 85 (libffi.dylib + 10757) [0x7ff81f6c1a05]
  30  -[NSApplication run] + 586 (AppKit + 193369) [0x7ff813d87359]
  30  -[NSApplication(NSEvent) _nextEventMatchingEventMask:untilDate:inMode:dequeue:] + 1411 (AppKit + 249692) [0x7ff813d94f5c]
  30  _DPSNextEvent + 886 (AppKit + 256240) [0x7ff813d968f0]
  30  _BlockUntilNextEventMatchingListInModeWithFilter + 70 (HIToolbox + 192213) [0x7ff819f88ed5]
  30  ReceiveNextEventCommon + 284 (HIToolbox + 192520) [0x7ff819f89008]
  30  RunCurrentEventLoopInMode + 292 (HIToolbox + 193505) [0x7ff819f893e1]
  30  CFRunLoopRunSpecific + 563 (CoreFoundation + 517421) [0x7ff81136b52d]
  30  __CFRunLoopRun + 1985 (CoreFoundation + 521135) [0x7ff81136c3af]
  30  __CFRunLoopDoTimers + 307 (CoreFoundation + 626157) [0x7ff811385ded]
  30  __CFRunLoopDoTimer + 927 (CoreFoundation + 627342) [0x7ff81138628e]
  30  ffi_closure_unix64 + 72 (libffi.dylib + 11164) [0x7ff81f6c1b9c]
  30  ffi_closure_unix64_inner + 506 (libffi.dylib + 9701) [0x7ff81f6c15e5]
  30  closure_fcn + 558 (Maestral + 5007294) [0x10502f7be]
  30  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  30  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  30  _PyEval_EvalFrameDefault + 28832 (Maestral + 3321360) [0x104e93e10]
  30  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  30  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  30  _PyEval_EvalFrameDefault + 28832 (Maestral + 3321360) [0x104e93e10]
  30  cfunction_vectorcall_FASTCALL_KEYWORDS + 132 (Maestral + 2648484) [0x104def9a4]
  30  context_run + 276 (Maestral + 3429492) [0x104eae474]
  30  _PyObject_MakeTpCall + 375 (Maestral + 2383047) [0x104daecc7]
  30  task_step + 886 (Maestral + 3757782) [0x104efe6d6]
  30  gen_send_ex + 468 (Maestral + 2457412) [0x104dc0f44]
  30  _PyEval_EvalFrameDefault + 11833 (Maestral + 3304361) [0x104e8fba9]
  30  gen_send_ex + 468 (Maestral + 2457412) [0x104dc0f44]
  30  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  30  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  30  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  30  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  30  _PyEval_EvalFrameDefault + 28002 (Maestral + 3320530) [0x104e93ad2]
  30  call_function + 624 (Maestral + 3332544) [0x104e969c0]
  30  _PyObject_MakeTpCall + 375 (Maestral + 2383047) [0x104daecc7]
  30  type_call + 340 (Maestral + 2743252) [0x104e06bd4]
  30  slot_tp_init + 213 (Maestral + 2778213) [0x104e0f465]
  30  _PyObject_Call_Prepend + 139 (Maestral + 2385931) [0x104daf80b]
  30  _PyObject_FastCallDictTstate + 173 (Maestral + 2382493) [0x104daea9d]
  30  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  30  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  30  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  30  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  30  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  27  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  27  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  27  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  27  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  27  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  27  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  22  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  22  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  22  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  _PyFunction_Vectorcall + 256 (Maestral + 2384960) [0x104daf440]
  22  _PyEval_EvalCode + 2852 (Maestral + 3336116) [0x104e977b4]
  22  _PyEval_EvalFrameDefault + 27654 (Maestral + 3320182) [0x104e93976]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  method_vectorcall + 204 (Maestral + 2394364) [0x104db18fc]
  22  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  22  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  22  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  22  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  22  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  22  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  22  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  18  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  18  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  18  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  15  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  15  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  15  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  11  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  11  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  11  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  10  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  10  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  10  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  6  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  6  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  6  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  4  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  4  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  4  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  3  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  3  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  3  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  2  _PyEval_EvalFrameDefault + 27625 (Maestral + 3320153) [0x104e93959]
  2  call_function + 411 (Maestral + 3332331) [0x104e968eb]
  2  function_code_fastcall + 229 (Maestral + 2385205) [0x104daf535]
  1  _PyEval_EvalFrameDefault + 15554 (Maestral + 3308082) [0x104e90a32]

[...]

madrobby avatar Oct 31 '21 17:10 madrobby

I'm now trying to do a clean selective sync, having unlinked the auth and grabbing stuff into a different folder. Note that I've two Maestral icons shown in the menubar... I can't remove or interact with the inactive one Screen Shot 2021-10-31 at 10 45 11 AM .

madrobby avatar Oct 31 '21 17:10 madrobby