datahub
datahub copied to clipboard
Don't put any sqlite databases on NFS
sqlite and NFS don't get along very well. However, Jupyter Notebooks uses several sqlite under $HOME by default. Our $HOME is on NFS, and this leads to weird problems.
The two major offenders are:
- IPython's HistoryManager. This is similar to
~/.bash_history
, but for IPython terminal work. Our users barely use the IPython console, so we can disable this. - Jupyter's Notebook Notary. This is used to trust notebooks that aren't created by the user, and can execute JS in their outputs. You need to explicitly 'trust' the notebook before this works. Unfortunately this also puts an sqlite database on NFS, which isn't something we want to do. If we disable this, users will have to explicitly trust a notebook to run JS outputs in it. This is also barely used, so we're good to remove this.
To disable these, we need to:
- Add an ipython_config.py file
- Add a jupyter_notebook_config.py file
- Put them in the right place
We should do this for all the images. We should be able to automate this at the helm level possibly?
@yuvipanda I believe this is already in place via directives in Dockerfiles, so while I'm ok with pursuing a strategy of automating this in Helm I can also do a quick check of the contents of the Dockerfiles and these files to make sure the proper configs are in place.
We already do this for notebook notary with https://github.com/berkeley-dsep-infra/datahub/blob/4f10eaea4d305f3364a3e92de388b37704bdec08/hub/values.yaml#L75. #2614 does this for historymanager too. Once that is merged we can close this.