Scrolling over notebook got unexpected jump backs

[AllanChain]

Description

With jupyterlab-myst installed, scrolling over the notebook is not smooth. Specifically, when I try to scroll up, it can sometimes jump back, and I have to scroll multiple times to get to the place I want. Like the following screen recording:

https://github.com/executablebooks/jupyterlab-myst/assets/36528777/2ae6b7f0-1206-42cb-b9f3-e129d29ad334

Without jupyterlab-myst installed, I can scroll smoothly without issue.

Reproduction steps:

1. Create a new venv

2. Install jupyterlab and jupyterlab-myst

3. Launch JLab and create a notebook

4. Put something in and make it scrollable

5. Scroll to the bottom and slowly scrolling up

Proposed solution

Sorry, I don't have any idea what's causing this issue.

Additional notes

Maybe it is related to https://github.com/jupyterlab/jupyterlab/issues/15795, but I can't reproduce that issue without jupyterlab-myst installed.

Environment:

• Browser: Firefox and Chromium
• JupyterLab: 4.2.2
• jupyterlab-myst: 2.4.2

[agoose77]

Thanks for this report. Looks like it might be a performance issue of some kind. I'll dig in to this.

[krassowski]

I can reproduce this. Related to https://github.com/jupyterlab/jupyterlab/issues/16326

The problem is that myst changes height of the cell after it is attached; this pushes the viewport down, which in turns detaches the cell, preventing user from scrolling up. There may be a way to fix it upstream in JupyterLab.

In the meantime a simple solution would be to cache the rendered content so that it is shown immediately when cell gets attached. It seems that the markdown cells when rendered with myst start from a blank state and are populated afterwards.

To make it easier to reproduce I would suggest using a markdown cell with admonition and a mermaid diagram.

[earlbellinger]

Having recently developed course materials using jupyterlab-myst, I am experiencing this bug quite frequently. I hope this can be resolved!

[nthiery]

Same here; it's been one of the main UI annoyances that was reported by our students.

[psychemedia]

I'm also trying to push MyST in notebooks but if the myst extension is the one upsetting notebook rendering I may have to remove it and downgrade all the MyST upgrades I added into notebooks till next presentation of course in October 2025.

[agoose77]

It's a real shame that this is causing regressions for people. There are definitely some bugs that we can fix here. The notebook code is quite hard to architect around, so it might take some time. But I am going to dedicate some cycles to this.

[psychemedia]

@agoose77 thanks.. let me know if there's anything I can hlep look out for or test

[krassowski]

I would appreciate some testing with https://github.com/jupyterlab/jupyterlab/pull/16950. I anticipate that this might not fully solve the problem, but believe it should alleviate it at least partially.

[AllanChain]

I have tried https://github.com/jupyterlab/jupyterlab/pull/16950, but I think the problem is still there. With the same notebook as the previously attached video, I got jumps at the same position with or without https://github.com/jupyterlab/jupyterlab/pull/16950. I tried another notebook (Markdown version attached below), without https://github.com/jupyterlab/jupyterlab/pull/16950, I got jumps both at the bottom and around the top of !pip list block. With https://github.com/jupyterlab/jupyterlab/pull/16950, I only got jumps around the top of !pip list block.

Notebook example

---

title: Test document author: Allan Chain

a = 42

:::{tip} Example : Tip of the day {eval}a :::

The Zen of Python, by Tim Peters

Beautiful is better than ugly.
Explicit is better than implicit.
Simple is better than complex.
Complex is better than complicated.
Flat is better than nested.
Sparse is better than dense.
Readability counts.
Special cases aren't special enough to break the rules.
Although practicality beats purity.
Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.
There should be one-- and preferably only one --obvious way to do it.
Although that way may not be obvious at first unless you're Dutch.
Now is better than never.
Although never is often better than *right* now.
If the implementation is hard to explain, it's a bad idea.
If the implementation is easy to explain, it may be a good idea.
Namespaces are one honking great idea -- let's do more of those!

!pip list

Package                   Version
------------------------- --------------
anyio                     4.6.2.post1
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 2.4.1
async-lru                 2.0.4
attrs                     24.2.0
babel                     2.16.0
beautifulsoup4            4.12.3
bleach                    6.2.0
certifi                   2024.8.30
cffi                      1.17.1
charset-normalizer        3.4.0
comm                      0.2.2
debugpy                   1.8.8
decorator                 5.1.1
defusedxml                0.7.1
executing                 2.1.0
fastjsonschema            2.20.0
fqdn                      1.5.1
h11                       0.14.0
httpcore                  1.0.6
httpx                     0.27.2
idna                      3.10
ipykernel                 6.29.5
ipython                   8.29.0
isoduration               20.11.0
jedi                      0.19.1
Jinja2                    3.1.4
json5                     0.9.25
jsonpointer               3.0.0
jsonschema                4.23.0
jsonschema-specifications 2024.10.1
jupyter_client            8.6.3
jupyter_core              5.7.2
jupyter-events            0.10.0
jupyter-lsp               2.2.5
jupyter_server            2.14.2
jupyter_server_terminals  0.5.3
jupyterlab                4.3.0
jupyterlab_myst           2.4.2
jupyterlab_pygments       0.3.0
jupyterlab_server         2.27.3
MarkupSafe                3.0.2
matplotlib-inline         0.1.7
mistune                   3.0.2
nbclient                  0.10.0
nbconvert                 7.16.4
nbformat                  5.10.4
nest-asyncio              1.6.0
notebook_shim             0.2.4
overrides                 7.7.0
packaging                 24.2
pandocfilters             1.5.1
parso                     0.8.4
pexpect                   4.9.0
pip                       24.2
platformdirs              4.3.6
prometheus_client         0.21.0
prompt_toolkit            3.0.48
psutil                    6.1.0
ptyprocess                0.7.0
pure_eval                 0.2.3
pycparser                 2.22
Pygments                  2.18.0
python-dateutil           2.9.0.post0
python-json-logger        2.0.7
PyYAML                    6.0.2
pyzmq                     26.2.0
referencing               0.35.1
requests                  2.32.3
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.21.0
Send2Trash                1.8.3
setuptools                75.3.0
six                       1.16.0
sniffio                   1.3.1
soupsieve                 2.6
stack-data                0.6.3
terminado                 0.18.1
tinycss2                  1.4.0
tornado                   6.4.1
traitlets                 5.14.3
types-python-dateutil     2.9.0.20241003
uri-template              1.3.0
urllib3                   2.2.3
wcwidth                   0.2.13
webcolors                 24.8.0
webencodings              0.5.1
websocket-client          1.8.0

[krassowski]

Thanks!

One idea for a workaround is to not detach cells when the change in cell heights is caused not by user action (execution) but by cell re-rendering and changing its height - and only attach cells in that case (if there was a long cell and it is now very short user needs to see cells below).

I also explored pinning the height of a cell as it enters the viewport for some time but it is hard to implement without side effects.

[agoose77]

Thanks for your investigations here @krassowski. In general, I want to do another pass at how we integrate the MyST renderer with the notebook/cell widget; right now, it feels very hard to cleanly connect the two due to the private implementation of parts of the widgets. We plan to get around to this very soon.

[krassowski]

I verified that a three-line change in JupyterLab "solves" the problem:

protected detachWidget(index: number, widget: Widget): void {
     // ...
+   if (cell.model.sharedModel.cell_type === 'markdown') {
+    this._toggleSoftVisibility(widget, false);
+    return;
+  }
     // ...

This builds on my previous fix to windowing in which I introduced a concept of "soft-hiding" active cell, this is instead of using display: none hiding it with opacity and height tricks; that allows the active cell to receive input when user scrolled away. This also means that jupyterlab-myst will not trigger re-rendering the cell (and thus changing height) when cell is unhidden.

Here is a proposal: we add a setting which defines which cells should be "hard-hidden" in full windowing mode. jupyterlab-myst could then decide to:

• a) implement my original suggestion of not resizing the cell when it gets rendered (i.e. cache previous renders), which will be much nicer for users anyways, or
• b) toggle the Notebook setting so that markdown cells are only ever "soft-hidden".

I might be able to open a PR for (b) once relevant change lands upstream but I really think that (a) is the better approach; that said I see that development in this repo stopped almost a year ago, so I suspect (a) would not be coming any time soon. If I open a PR for (b) is there someone who could review and release it?

[krassowski]

we add a setting which defines which cells should be "hard-hidden" in full windowing mode

I expanded on the idea in https://github.com/jupyterlab/jupyterlab/issues/17331 though it seems it would better if it was a new optional property on Renderer rather than a new setting. jupyterlab-myst would specify it as an extra property on:

https://github.com/jupyter-book/jupyterlab-myst/blob/4c70b0312aee6f2f65c18d1aeec80521c1fa5bc3/src/mime.tsx#L10-L13

[krassowski]

Oh, that would not suffice for markdown cells because this extension does not actually swap the public renderer, but only overwrites the render() method:

https://github.com/jupyter-book/jupyterlab-myst/blob/4c70b0312aee6f2f65c18d1aeec80521c1fa5bc3/src/MySTMarkdownCell.tsx#L51-L54

https://github.com/jupyter-book/jupyterlab-myst/blob/4c70b0312aee6f2f65c18d1aeec80521c1fa5bc3/src/MySTMarkdownCell.tsx#L206-L220

I guess that can be adjusted.

[agoose77]

Thanks @krassowski for your efforts on this. We are a little bandwidth limited on parts of the stack at the moment, so it takes a little longer to circle around than is ideal.

I'll take a moment to engage with your last point -- the MyST extension has to do a fair amount of work to fight the restricted interface and design assumptions of the notebook renderer, and it's not perfect. We might want to redesign parts of this and hopefully improve things.

[krassowski]

If you could write down a proposal on what should be changed in upstream API to make it easier for jupyterlab-myst to swap a renderer without hacks this would be super useful. If there is already an issue feel free to tag me.

[krassowski]

Hi there, JupyterLab 4.4.0 has a special-case for jupyterlab-myst, upgrading should help you all.

[krassowski]

Any luck testing JupyterLab 4.4.0 @AllanChain @nthiery @earlbellinger?

[AllanChain]

The scrolling in JupyterLab 4.4.0 is smooth, and I haven't observed jumps when scrolling.

[nthiery]

Dear @krassowski, I don't have a serious test case, but from the little test I just did, scrolling seems to be smooth. I'll now observe how things go over my student shoulders. Thanks a lot!

[ELind77]

I'm still getting lag and jumps while scrolling after upgrading to JupyterLab 4.4.0 (not clear if it's the same issue or something new though).

I'm also experiencing cases where I'll scroll down a page and the bottom part of the notebook viewport is empty/white, no cells rendered at all and if I keep scrolling, cells pop in. After skimming this thread before deciding to upgrade for 4.4.0, I saw the discussion about cells being detached and reattached and it made me think that there's a bug somewhere in the logic that determines what's in the viewport and what order cells are in. As an additional piece of evidence, I've started encountering a bug in longer notebooks where navigating cells with the arrow keys doesn't take me in order and trying to add a cell below the selected cell will add it below a different cell. I saw another issue here with a similar title. All of these point to a bug in whatever logic is trying to modify the DOM (imo).

[dustinfreeman]

Anecdotally, I'm observing jumpy scrolling while python in processing in JupyterLab 4.4.3. I can't determine if this is the same bug resolved in #17339 or something else