holoviz
Pyodide Conversion: fix requirements handling
This fixes: #6787 AND #4164 and adjusts the documentation
Why both? adding custom wheels as requirements requires making those available to the Pyodide environment at runtime. One could place them into the output structure as wheels to be served with the app, but this might clutter it a lot if there are many wheels (and also clash with any bundling system to be implemented). Also semantics with this solution seem clearer: you convert a panel app requiring custom wheels, you get a zip with dependencies and you distribute the app and the zip-file.
As the infrastructure for fixing #4164 was in place with this solution, I then also added a --resources-flag to the convert-command to include custom resources (such as template files/data files/...) required by the custom-python-portion of the panel app. Everything is packed into a single zipfile, because that's easiest to use with the pyscript config (which only allows to uncompress one zip to a specific wildcard-location); also I bumped the pyscript version, because that feature only got added with one of the small releases.
Currently BROKEN: on main, someone updated the version number in package.json which is used for getting the CDN_DIST-url. I think PY_VERSION in panel/io/convert.py (which references a python __version__ should also be updated accordingly and possibly also be acquired from a single-source-of-truth!
Reasons for restructuring:
- installing the dependencies one-by-one with micropip for the pyodide-worker seems clever and gives some responsiveness. However this doesn't use custom packages installed in previous calls for dependency resolution (...) and thus does not work if you want to override specific dependencies with custom-builds!
- I split the code for finding the requirements into a function
collect_python_requirements(finding the requirements or parsing the requirements-file) and some code rewriting/collecting filesystem-URLs.script_to_htmlnow takes a micropip-ready list of package-specifications!
Small things:
- I would put the documentation for standalone-usage of panel in WASM in front of the conversion docs. Currently this is very weird clicking through, showing a conversion tool and then showing how to manually write your code.
- I replaced some very inconsistent usage of single and doublequotes
- as http-patch is in the pyodide-lockfile (from which micropip installs WASM-wheels) and this only ever includes one version (for a specific version of Pyodide), it doesn't make sense to version-lock this. Also removed the
MIN_VERSIONSfor this reason. Once pyodide implements proper requirements handling this should be put into the requirements-file like for every other project.
This is incredible, thanks for diving into this so deeply @flxmr! I'll have to dig into this a little bit to unpack it and I'm out until next week but would like to target this for the 1.5 release in early to mid June.
I'm really sorry this has sat around for so long. It's a great contribution and should have gone in long ago. I didn't want to burden you with having to resolve the conflicts so I've restarted this effort in https://github.com/holoviz/panel/pull/8181, preserving your commits.