Editing Jupyter Notebook in RTC mode will not update checkpoint, keeps changing to "not trusted" and reverts to previous state

[jesssssssssssssssssss]

Description

When editing a notebook with RTC enabled, the notebook will randomly revert to the last checkpoint (which was 7 days ago in this case) and all changes since then are lost. Neither Edit -> Undo or CTRL+Z will restore changes. Running my notebook requires VPN connection which I think might be part of the problem as the notebook will periodically change from "trusted" to "not trusted". When I then go to File -> Trust Notebook, it says something about the version I have open is not the same as the one on disk and asks me if I want to revert to the last checkpoint or overwrite the copy on disk. I always opt to overwrite with the version I have open, yet the checkpoint does not update and the issue persists. There seems to be no way to manually save the notebook other than downloading and re-opening it. I've opened and edited the document at some point in JupyterLab and not had this exact problem, but in doing this I have noticed another strange bug where I'll close and shut down the notebook, and upon reopening it, all cells in the notebook are duplicated (as in cells 1-20 appear, then at the end of the notebook there's cells 1-20 again). I've also copied a link to the document and opened it in an Incognito browser (Google Chrome) to test RTC when I first enabled it, but I closed and shut down the notebook before closing the browser.

Reproduce

I can't seem to figure out exactly what I'm doing to cause this behavior but here is essentially what's happened:

1. Open document in Jupyter Notebook (while connected to VPN): http://localhost:8888/notebooks/RTC:Documents/file.ipynb
2. Everything is fine, notebook is trusted, running individual cells works as normal.
3. Open an instance of JupyterLab while I still had notebook open in a separate tab
4. Shut down JupyterLab
5. Return to the notebook tab
6. All recent changes lost
7. Restore changes and download notebook (since there is no way to manually save)
8. Open downloaded copy of notebook in Jupyter Notebook
9. Make a bunch of edits and run all cells
10. Computer locked while notebook was running (possibly disconnecting me from the VPN and/or interrupting kernel)
11. Unlock computer
12. All recent changes lost again

Context

Operating System and version: Microsoft Windows [Version 10.0.19044.3086] Browser and version: Chrome Version 116.0.5845.188 IPython : 8.15.0 ipykernel : 6.25.2 ipywidgets : 8.1.0 jupyter_client : 8.3.1 jupyter_core : 5.3.1 jupyter_server : 2.7.3 jupyterlab : 4.0.5 nbclient : 0.8.0 nbconvert : 7.8.0 nbformat : 5.9.2 notebook : 7.0.3 qtconsole : 5.4.4 traitlets : 5.9.0

Related/similar issues:

https://github.com/jupyterlab/jupyterlab/issues/10852 https://github.com/jupyterlab/jupyterlab/issues/11348 https://github.com/jupyter/notebook/issues/484 https://github.com/jupyterlab/jupyter-collaboration/issues/163

Cross-listed here:

https://github.com/jupyter/notebook/issues/7065

[welcome[bot]]

Thank you for opening your first issue in this project! Engagement like this is essential for open source projects! :hugs:
If you haven't done so already, check out Jupyter's Code of Conduct. Also, please try to follow the issue template as it helps other other community members to contribute more effectively. You can meet the other Jovyans by joining our Discourse forum. There is also an intro thread there where you can stop by and say Hi! :wave:
Welcome to the Jupyter community! :tada:

[navdeepgill14]

Hello,

I hope this message finds you well. I've been closely following the ongoing issue, and it seems like it hasn't been resolved yet. I'm more than willing to step in and take over resolving it. Please consider assigning me to the task, and I'll do my best to ensure a swift and effective resolution.

Thank you for your attention to this matter.

Best regards, Navdeep

[davidbrochart]

I think checkpointing should be disabled altogether in RTC, since there is the same kind of functionality in the YStore (by default, a .jupyter_ystore.db in the current directory).

[jesssssssssssssssssss]

I'm now having this issue, not sure if it's related or not but here's what's happening:

1. Turn on computer and connect to VPN
2. Connect to Jupyter server, which brings me to Jupyter Lab
3. Try to open a notebook but it throws the warning in the attached screenshot and won't open, pressing "Ok" dismisses the warning but it continues to pop up until I close the notebook I was trying to open
4. Close all tabs, notebooks, terminals, etc and check "Running Terminals and Kernals" tab to make sure it's not already open/running somewhere but it doesn't show anything, clicked all three "Shut Down All" buttons just in case
5. Tried to open the notebook again but still got the same warning, notebook couldn't open
6. Went to File -> Hub Control Panel to stop my server
7. Started my server again which re-opened Jupyter Lab
8. Notebook I just tried to open (without success) is now open along with another notebook, an html file, and a terminal window (so 4 tabs in total, both notebooks are connected to kernels and are showing in the list of running kernels - see second attached screenshot) upon start up even though I had just closed and shut everything down before stopping my server.
9. Edit the notebook as usual now that it has opened, everything is fine until I close the notebook tab
10. Attempting to re-open the notebook sometime later brings me back to step 1

[larrygao001]

We are experiencing similar issues from multiple users when using Jupyterlab in RTC mode:

1. All notebook cells got duplicated.
2. Significant part of the code disappeared.
3. Recent checkpoints vanished, notebook cannot be recovered because the last checkpoint is from 17 days ago even though the users were actively working on the notebook.

The issues happened intermittently and we haven't found a way to reproduce it. Given that they were also reported previously by multiple people, can you prioritize the investigation?

Context:

Jupyterlab 3.6.5 jupyter_server 2.0.0rc3 jupyter_server_ydoc 0.8.0 ypy_websocket 0.8.4 tornado 6.1.0

[davidbrochart]

Checkpoints are not used when RTC is enabled. Are you mixing RTC and non-RTC modes in any way?

[nsauerwein]

Hey all,

I am also having the same issue. It is quite problematic as it makes us loose data and code. We are running jupyterlab in on a jupyterhub server running Fedora.

Sometimes the files become so long that the computer struggles to open them. I am not sure whether this is related, but jupyterlab always asks whether you want to override the file on the disk. I tried to change the delay accepted for modification date. Unfortunately without success.

Thanks in advance for you help!