jupyterlite
Reduce the initial download size
Problem
On the demo website the xeus-python files are a bit over 10MB in size:
This size can grow quite significantly as more packages are added by default. For example:
Proposed Solution
Not sure how feasible it is, but maybe there is a way to trim that down a little bit.
Additional context
Also raised by @jasongrout in https://github.com/jupyter-widgets/ipywidgets/pull/3491#issuecomment-1177391508.
Even if the data was somehow chunked, so only the packages that were actually imported were downloaded, would be great.
As mentioned above, python_data.data size goes up quickly when adding more packages. For example the command below creates a python_data.data file which is 117MB big which only a few "reasonably standard" packages (scikit-learn,matplotlib-base,seaborn,plotly):
jupyter lite build --XeusPythonEnv.packages=scikit-learn,matplotlib-base,seaborn,plotly
In practice it means that you can not use github pages to deploy your site, since there is a 100MB per-file limit.
I am aware you can use alternative options (e.g. Netlify) but I would guess that pushing into github pages is the simple setup and is likely to be used by a significant portion of people.
Also, for set-ups that push into a gh-pages branch (or .github.io repo) on each commit to main (scikit-learn does that but I am guessing others do as well), even if we are below 100MB, it also means the github repo size may grow quickly, which makes it a bit painful to interact with (e.g. long git commands).
I already implemented such chunking, this will be added in one of the next releases
Great to hear that, just curious (and to get a better idea of the landscape), in one of the next releases of which project? jupyterlite, emscripten-forge, jupyterlite-xeus-python, something else?
This was done in empack, the tool that we use for "packing" the Python environment, we still need to make use of it in jupyterlite-xeus-python.
