Sub-issue: Drop requirement on `faiss-cpu`

[dlqqq]

trafficstars

Problem

Listing faiss-cpu as a required dependency has caused numerous issues for users.

• Meta does not distribute any builds of FAISS. FAISS distributions on PyPI and Conda Forge are unofficially maintained by third-party contributors.
• On PyPI, we can only install FAISS via faiss-cpu, therefore we have faiss-cpu listed as a required dependency.
• However, installing faiss-cpu from Conda Forge does not install a Python package named faiss-cpu, but instead a Python package named faiss. This causes pip check to fail, since faiss-cpu is not installed. Jupyter AI still works however, since both packages provide the faiss module that we import.
• FAISS releases on PyPI and Conda Forge may lag behind the latest releases of FAISS on GitHub, as they are maintained by third-party contributors. This will limit the rate at which we can upgrade to new releases of Python, e.g. Python 3.13.

Proposed Solution

• Make faiss-cpu an optional dependency.
• Allow other vector databases to be used by /learn.
• Ideally, users should be able to install support for other vector databases at runtime, without requiring a restart of the Jupyter Server.

Additional context

• https://github.com/conda-forge/jupyter-ai-feedstock/pull/47