`micropip.install("<...>")` does not uphold custom PyPI indices

[agriyakhetarpal]

A description of the issue

Last year, https://github.com/pyodide/micropip/pull/74/ added support for setting a custom index URL that adheres to the PEP-503-based simple repository APIs for the purpose of installing files from an index different from PyPI.

However, I cannot seem to make this work while setting up interactive examples on Read the Docs for the development version of a package, here is a reproducer: https://github.com/PyWavelets/pywt/pull/728

Some steps to reproduce

In the Examples section on the Read the Docs PR preview for a simple API reference, say, here: https://pywavelets--728.org.readthedocs.build/en/728/ref/wavelets.html#wavelet-families

I am trying to run this code snippet inside a code block in the Pyodide-enabled JupyterLite notebook (that can be opened with the "Try it in your browser" button):

import micropip
await micropip.install("pywavelets", index_urls="https://pypi.anaconda.org/scientific-python-nightly-wheels/simple")

and I even tried this:

micropip.set_index_urls(urls="https://pypi.anaconda.org/scientific-python-nightly-wheels/simple")
await micropip.install("pywavelets")

but micropip.list() still prints out (note: truncated output):

PyWavelets         | 1.4.1   | pyodide

rather than the version I am trying to install (1.6.0rc1 at the time of writing), which doesn't make sense – so I tried reloading the package with importlib.reload() and importlib.invalidate_caches(), but to no avail – it is still 1.4.1. This suggests that the micropip.install() invocation might be failing silently.

Expected behaviour

It would be great to see the development version of PyWavelets getting downloaded and installed (i.e., v1.6.0). For interactive examples using jupyterlite-pyodide-kernel, it is not possible to pre-install wheels (unlike xeus-kernel, where pre-loading wheels is possible through emscripten-forge), so we are adding an admonition that displays the command to execute an installation of the wheels for interested users.

---

P.S. A workaround I am considering is to download the WASM wheel by overriding the Read the Docs documentation build process, place it into an appropriate location as listed in https://jupyterlite.readthedocs.io/en/latest/howto/pyodide/wheels.html, and proceed to install from the wheel file from there. However, being a workaround – this has negative implications and is too hacky to use, since it would be expected for the functionality added to just work.