Stuck git lockfile causes severe delays to IO operations in JupyterLab

[krassowski]

Description

Sometimes the git lock file gets stuck, because the process that was locking was terminated and it was not removed properly. This is not a fault of this extension - just happens. However, when such a lockfile file exists:

1. the refresh operations get stuck too and in a weird consequence saving files from JupyterLab UI can take anywhere from seconds to dozens of seconds (rather than instant).
2. the GUI does not indicate what is going on. Only manual use of git reveals the problem and it is gone once the lockfile is removed. The extension errors are not presented to the user, so they are left puzzled with why their files stopped saving immediately (I was genuinely concerned that my hard drive may be off because it persisted after restart).

Reproduce

1. Open JupyterLab with git extension installed
2. Open a file, (modify it and save it) x few times
3. See that the save time is instant
4. Run touch .git/index.lock in terminal
5. See that no indication of an issue is shown in GUI
6. Repeat (2)
7. See that files take a long time to be saved ("Saving started" shows quickly on the status bar, but you need to wait for "Saving completed" a long time). It may not happen immediately, but only after a few modify - save cycles.
8. See the errors in the browser console, including serverconnection.js:205 POST http://localhost:8891/git/status?1608467764831 500 (Internal Server Error) (see Web Browser Output for detailed tracback)

Expected behavior

1. The lockfile should not block the IO
2. The interface should indicate that lockfile prevents normal operations and that a user action is needed

Context

• Python package version: 0.23.2
• Extension version:

JupyterLab v2.2.8
Known labextensions:
   app dir: [...]/share/jupyter/lab
        @ijmbarr/jupyterlab_spellchecker v0.2.0  enabled  OK
        @jupyterlab/git v0.23.2  enabled  OK
        @jupyterlab/toc v4.0.0  enabled  OK
        @krassowski/jupyterlab-lsp v2.1.1  enabled  OK
        nbdime-jupyterlab v2.0.1  enabled  OK

• Git version: 2.27.0
• Operating System and its version: Ubuntu 20.10

Web Browser Output

POST http://localhost:8891/git/status?1608467764831 500 (Internal Server Error)
n @ serverconnection.js:205
n @ serverconnection.js:76
S @ git.js:46
(anonymous) @ model.js:752
execute @ taskhandler.js:58
refreshStatus @ model.js:751
async function (async)
refreshStatus @ model.js:741
(anonymous) @ index.js:93
m @ index.es6.js:459
c @ index.es6.js:416
e.emit @ index.es6.js:99
_onFileChanged @ model.js:506
m @ index.es6.js:459
c @ index.es6.js:416
e.emit @ index.es6.js:99
_onFileChanged @ index.js:411
m @ index.es6.js:459
c @ index.es6.js:416
e.emit @ index.es6.js:99
save @ index.js:655
async function (async)
save @ index.js:647
save @ index.js:284
(anonymous) @ context.js:558
Promise.then (async)
_maybeSave @ context.js:542
_save @ context.js:445
async function (async)
_save @ context.js:443
(anonymous) @ context.js:200
Promise.then (async)
save @ context.js:199
execute @ index.js:452
e.execute @ index.es6.js:357
e._executeKeyBinding @ index.es6.js:517
e.processKeydownEvent @ index.es6.js:456
e.evtKeydown @ index.es6.js:355
e.handleEvent @ index.es6.js:311
model.js:769 Error: Unable to get the lock on the directory
    at S (git.js:65)
    at async http:/localhost:8891/static/lab/vendors~main.b31f19a7e614c4b9ecbc.js:2
    at async Et.execute (taskhandler.js:58)
    at async Tt.refreshStatus (model.js:751)
refreshStatus @ model.js:769
async function (async)
refreshStatus @ model.js:741
(anonymous) @ index.js:93
m @ index.es6.js:459
c @ index.es6.js:416
e.emit @ index.es6.js:99
_onFileChanged @ model.js:506
m @ index.es6.js:459
c @ index.es6.js:416
e.emit @ index.es6.js:99
_onFileChanged @ index.js:411
m @ index.es6.js:459
c @ index.es6.js:416
e.emit @ index.es6.js:99
save @ index.js:655
async function (async)
save @ index.js:647
save @ index.js:284
(anonymous) @ context.js:558
Promise.then (async)
_maybeSave @ context.js:542
_save @ context.js:445
async function (async)
_save @ context.js:443
(anonymous) @ context.js:200
Promise.then (async)
save @ context.js:199
execute @ index.js:452
e.execute @ index.es6.js:357
e._executeKeyBinding @ index.es6.js:517
e.processKeydownEvent @ index.es6.js:456
e.evtKeydown @ index.es6.js:355
e.handleEvent @ index.es6.js:311

[fcollonval]

Thanks for reporting the issue @krassowski The interaction with the save feature is surprising and definitely should be addressed.

@mlucool Do you have any idea or pointers where to look for the issue?

[mlucool]

the refresh operations get stuck too and in a weird consequence saving files from JupyterLab UI can take anywhere from seconds to dozens of seconds (rather than instant).

My guess is it sounds like it's this sleep blocking. It's been a while since I looked at this, but I thought awaiting the sleep would be enough to keep this from blocking other actions taken on the server. Maybe we need move work to a thread?