jupyter_server icon indicating copy to clipboard operation
jupyter_server copied to clipboard

Published 20 hours ago • verified publisher icon jupyter-server

Contents API fetches are cached for too long

Open sebbov opened this issue 2 years ago • 1 comments

Description

Without cache directives in the response headers from the Contents API, Chrome appears to cache fetch responses for a duration proportional to the time since the reported last modification of the content. This prevents fetches from seeing updated contents when contents (on first fetch) haven't seen recent modifications.

Reproduce

Start the server:

jupyter server \
  --ServerApp.port=8892 \
  --ServerApp.tornado_settings='{"headers": { "Content-Security-Policy": "frame-ancestors self http://127.0.0.1:8892/; connect-src '"'"'self'"'"' " }}' \
  --ServerApp.token=''

From the server's root dir (same cwd as previous command) make a dir to list contents from. Pretend it was created a few months ago (but using just a few hours ago repros as well):

mkdir some_dir
touch -t 202301010000 some_dir

In an incognito Chrome window, clear browser cache, then navigate to the server root url (http://127.0.0.1:8892/). Be welcomed by "A Jupyter Server is running.".

In the Developer Tools console, exercise the Contents API for the created dir:

fetch('http://127.0.0.1:8892/api/contents/some_dir').then(res => res.json()).then(console.log)

Response headers should look like:

HTTP/1.1 200 OK
Server: TornadoServer/6.3.3
Content-Type: application/json
Date: Tue, 24 Oct 2023 18:10:12 GMT
X-Content-Type-Options: nosniff
Content-Security-Policy: frame-ancestors self http://127.0.0.1:8892/; connect-src 'self' ; default-src 'none'
Access-Control-Allow-Origin: *
Last-Modified: Sun, 01 Jan 2023 08:00:00 GMT
Etag: "d2879b009a893042a30501118aea5e98986eac8e"
Content-Length: 227

Run the fetch again and notice the response is obtained from disk cache.

Make a modification to the directory:

touch some_dir/a

Run the fetch again and notice the response is obtained from disk cache, and hence doesn't include some_dir/a. Repeatable for minutes.

Expected behavior

A fetch with default options should show the new directory entry. (Caching this for a few seconds seems acceptable, but not 10m+.)

The problem can be avoided by specifying cache use behavior in the fetch options (e.g. no-store). I don't know if it is expected to have to specify this in the frontend, vs. having the server control the caching behavior better. I found https://github.com/jupyter-server/jupyter_server/commit/fe730a60c2d829cfee8f62dff9ba312e258d2cf2 which disables caching for unversioned static files and suggests server-side handling was at least preferred there.

Trying a simple version of that:

diff --git a/jupyter_server/services/contents/handlers.py b/jupyter_server/services/contents/handlers.py
index 50e7703db..b193eb0d0 100644
--- a/jupyter_server/services/contents/handlers.py
+++ b/jupyter_server/services/contents/handlers.py
@@ -89,6 +89,9 @@ class ContentsHandler(ContentsAPIHandler):
             self.set_header("Location", location)
         self.set_header("Last-Modified", model["last_modified"])
         self.set_header("Content-Type", "application/json")
+        # disable browser caching, rely in 304 replies for savings
+        if "v" not in self.request.arguments:
+          self.add_header("Cache-Control", "no-cache")
         self.finish(json.dumps(model, default=json_default))
 
     @web.authenticated

, conditional requests are performed for every fetch, problem no longer observed. IDK enough about browser caching to know if this is the right fix though.

I haven't seen this Chrome behavior with page loads, only fetches.

Context

  • Operating System and version: Debian based
  • Browser and version: Chrome 118.0.5993.88
  • Jupyter Server version: head github repo
Troubleshoot Output 
$PATH:
	/home/seb/venv/jupyter_server.1/bin
	/usr/local/buildtools/java/jdk/bin
	/usr/local/sbin
	/usr/local/bin
	/usr/sbin
	/usr/bin
	/sbin
	/bin
	/home/seb/bin
	/home/seb/bin
	/home/seb/bin

sys.path:
/home/seb/venv/jupyter_server.1/bin
/usr/local/buildtools/current/sitecustomize
/usr/lib/python311.zip
/usr/lib/python3.11
/usr/lib/python3.11/lib-dynload
/home/seb/venv/jupyter_server.1/lib/python3.11/site-packages
/home/seb/src/github.com/jupyter_server/jupyter_server
/home/seb/src/github.com/jupyter_server/jupyter_server/examples/simple


sys.executable:
/home/seb/venv/jupyter_server.1/bin/python3


sys.version:
3.11.5 (main, Aug 25 2023, 07:43:52) [GCC 12.2.0]


platform.platform():
Linux-6.5.3-1rodete1-amd64-x86_64-with-glibc2.37


which -a jupyter:
/home/seb/venv/jupyter_server.1/bin/jupyter


pip list:
Package                   Version    Editable project location
------------------------- ---------- ---------------------------------------------------------------------------------------
anyio                     4.0.0
argon2-cffi               23.1.0
argon2-cffi-bindings      21.2.0
arrow                     1.3.0
asttokens                 2.4.0
attrs                     23.1.0
backcall                  0.2.0
beautifulsoup4            4.12.2
bleach                    6.1.0
certifi                   2023.7.22
cffi                      1.16.0
cfgv                      3.4.0
charset-normalizer        3.3.1
comm                      0.1.4
debugpy                   1.8.0
decorator                 5.1.1
defusedxml                0.7.1
distlib                   0.3.7
executing                 2.0.0
fastjsonschema            2.18.1
filelock                  3.12.4
flaky                     3.7.0
fqdn                      1.5.1
identify                  2.5.30
idna                      3.4
iniconfig                 2.0.0
ipykernel                 6.25.2
ipython                   8.16.1
isoduration               20.11.0
jedi                      0.19.1
Jinja2                    3.1.2
jsonpointer               2.4
jsonschema                4.19.1
jsonschema-specifications 2023.7.1
jupyter_client            8.4.0
jupyter_core              5.4.0
jupyter-events            0.8.0
jupyter_server            2.9.0.dev0 /home/seb/src/github.com/jupyter_server/jupyter_server
jupyter-server-example    0.0.1      /home/seb/src/github.com/jupyter_server/jupyter_server/examples/simple
jupyter_server_terminals  0.4.4
jupyterlab-pygments       0.2.2
MarkupSafe                2.1.3
matplotlib-inline         0.1.6
mistune                   3.0.2
nbclient                  0.8.0
nbconvert                 7.9.2
nbformat                  5.9.2
nest-asyncio              1.5.8
nodeenv                   1.8.0
overrides                 7.4.0
packaging                 23.2
pandocfilters             1.5.0
parso                     0.8.3
pexpect                   4.8.0
pickleshare               0.7.5
pip                       23.1.2
platformdirs              3.11.0
pluggy                    1.3.0
pre-commit                3.5.0
prometheus-client         0.17.1
prompt-toolkit            3.0.39
psutil                    5.9.6
ptyprocess                0.7.0
pure-eval                 0.2.2
pycparser                 2.21
Pygments                  2.16.1
pytest                    7.4.2
pytest-console-scripts    1.4.1
pytest-jupyter            0.7.0
pytest-timeout            2.2.0
python-dateutil           2.8.2
python-json-logger        2.0.7
PyYAML                    6.0.1
pyzmq                     25.1.1
referencing               0.30.2
requests                  2.31.0
rfc3339-validator         0.1.4
rfc3986-validator         0.1.1
rpds-py                   0.10.6
Send2Trash                1.8.2
setuptools                67.8.0
six                       1.16.0
sniffio                   1.3.0
soupsieve                 2.5
stack-data                0.6.3
terminado                 0.17.1
tinycss2                  1.2.1
tornado                   6.3.3
traitlets                 5.11.2
types-python-dateutil     2.8.19.14
uri-template              1.3.0
urllib3                   2.0.7
virtualenv                20.24.6
wcwidth                   0.2.8
webcolors                 1.13
webencodings              0.5.1
websocket-client          1.6.4

sebbov avatar Oct 24 '23 18:10 sebbov