`ModuleNotFoundError: No module named 'requests'` with fresh install

[aviflax]

I rebuilt my docker image this morning, and after I did so my application broke.

Here’s a reproduction:

➜  /tmp rm -Rf .venv
➜  /tmp python3 -m venv .venv
➜  /tmp source .venv/bin/activate
(.venv) ➜  /tmp pip freeze
(.venv) ➜  /tmp pip install llm
Collecting llm
  Using cached llm-0.11.1-py3-none-any.whl.metadata (6.0 kB)
Collecting click (from llm)
  Using cached click-8.1.7-py3-none-any.whl.metadata (3.0 kB)
Collecting openai (from llm)
  Using cached openai-1.0.1-py3-none-any.whl.metadata (16 kB)
Collecting click-default-group>=1.2.3 (from llm)
  Using cached click_default_group-1.2.4-py2.py3-none-any.whl.metadata (2.8 kB)
Collecting sqlite-utils>=3.35.0 (from llm)
  Using cached sqlite_utils-3.35.2-py3-none-any.whl.metadata (7.6 kB)
Collecting sqlite-migrate>=0.1a2 (from llm)
  Using cached sqlite_migrate-0.1b0-py3-none-any.whl.metadata (5.4 kB)
Collecting pydantic>=1.10.2 (from llm)
  Using cached pydantic-2.4.2-py3-none-any.whl.metadata (158 kB)
Collecting PyYAML (from llm)
  Using cached PyYAML-6.0.1-cp311-cp311-macosx_11_0_arm64.whl.metadata (2.1 kB)
Collecting pluggy (from llm)
  Using cached pluggy-1.3.0-py3-none-any.whl.metadata (4.3 kB)
Collecting python-ulid (from llm)
  Using cached python_ulid-2.2.0-py3-none-any.whl.metadata (3.9 kB)
Requirement already satisfied: setuptools in ./.venv/lib/python3.11/site-packages (from llm) (68.2.2)
Requirement already satisfied: pip in ./.venv/lib/python3.11/site-packages (from llm) (23.3.1)
Collecting annotated-types>=0.4.0 (from pydantic>=1.10.2->llm)
  Using cached annotated_types-0.6.0-py3-none-any.whl.metadata (12 kB)
Collecting pydantic-core==2.10.1 (from pydantic>=1.10.2->llm)
  Using cached pydantic_core-2.10.1-cp311-cp311-macosx_11_0_arm64.whl.metadata (6.5 kB)
Collecting typing-extensions>=4.6.1 (from pydantic>=1.10.2->llm)
  Using cached typing_extensions-4.8.0-py3-none-any.whl.metadata (3.0 kB)
Collecting sqlite-fts4 (from sqlite-utils>=3.35.0->llm)
  Using cached sqlite_fts4-1.0.3-py3-none-any.whl (10.0 kB)
Collecting tabulate (from sqlite-utils>=3.35.0->llm)
  Using cached tabulate-0.9.0-py3-none-any.whl (35 kB)
Collecting python-dateutil (from sqlite-utils>=3.35.0->llm)
  Using cached python_dateutil-2.8.2-py2.py3-none-any.whl (247 kB)
Collecting anyio<4,>=3.5.0 (from openai->llm)
  Using cached anyio-3.7.1-py3-none-any.whl.metadata (4.7 kB)
Collecting distro<2,>=1.7.0 (from openai->llm)
  Using cached distro-1.8.0-py3-none-any.whl (20 kB)
Collecting httpx<1,>=0.23.0 (from openai->llm)
  Using cached httpx-0.25.1-py3-none-any.whl.metadata (7.1 kB)
Collecting tqdm>4 (from openai->llm)
  Using cached tqdm-4.66.1-py3-none-any.whl.metadata (57 kB)
Collecting idna>=2.8 (from anyio<4,>=3.5.0->openai->llm)
  Using cached idna-3.4-py3-none-any.whl (61 kB)
Collecting sniffio>=1.1 (from anyio<4,>=3.5.0->openai->llm)
  Using cached sniffio-1.3.0-py3-none-any.whl (10 kB)
Collecting certifi (from httpx<1,>=0.23.0->openai->llm)
  Using cached certifi-2023.7.22-py3-none-any.whl.metadata (2.2 kB)
Collecting httpcore (from httpx<1,>=0.23.0->openai->llm)
  Using cached httpcore-1.0.1-py3-none-any.whl.metadata (20 kB)
Collecting six>=1.5 (from python-dateutil->sqlite-utils>=3.35.0->llm)
  Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Collecting h11<0.15,>=0.13 (from httpcore->httpx<1,>=0.23.0->openai->llm)
  Using cached h11-0.14.0-py3-none-any.whl (58 kB)
Using cached llm-0.11.1-py3-none-any.whl (36 kB)
Using cached click_default_group-1.2.4-py2.py3-none-any.whl (4.1 kB)
Using cached pydantic-2.4.2-py3-none-any.whl (395 kB)
Using cached pydantic_core-2.10.1-cp311-cp311-macosx_11_0_arm64.whl (1.7 MB)
Using cached sqlite_migrate-0.1b0-py3-none-any.whl (10.0 kB)
Using cached sqlite_utils-3.35.2-py3-none-any.whl (67 kB)
Using cached click-8.1.7-py3-none-any.whl (97 kB)
Using cached openai-1.0.1-py3-none-any.whl (153 kB)
Using cached pluggy-1.3.0-py3-none-any.whl (18 kB)
Using cached python_ulid-2.2.0-py3-none-any.whl (9.4 kB)
Using cached PyYAML-6.0.1-cp311-cp311-macosx_11_0_arm64.whl (167 kB)
Using cached annotated_types-0.6.0-py3-none-any.whl (12 kB)
Using cached anyio-3.7.1-py3-none-any.whl (80 kB)
Using cached httpx-0.25.1-py3-none-any.whl (75 kB)
Using cached tqdm-4.66.1-py3-none-any.whl (78 kB)
Using cached typing_extensions-4.8.0-py3-none-any.whl (31 kB)
Using cached certifi-2023.7.22-py3-none-any.whl (158 kB)
Using cached httpcore-1.0.1-py3-none-any.whl (76 kB)
Installing collected packages: sqlite-fts4, typing-extensions, tqdm, tabulate, sniffio, six, PyYAML, python-ulid, pluggy, idna, h11, distro, click, certifi, annotated-types, python-dateutil, pydantic-core, httpcore, click-default-group, anyio, sqlite-utils, pydantic, httpx, sqlite-migrate, openai, llm
Successfully installed PyYAML-6.0.1 annotated-types-0.6.0 anyio-3.7.1 certifi-2023.7.22 click-8.1.7 click-default-group-1.2.4 distro-1.8.0 h11-0.14.0 httpcore-1.0.1 httpx-0.25.1 idna-3.4 llm-0.11.1 openai-1.0.1 pluggy-1.3.0 pydantic-2.4.2 pydantic-core-2.10.1 python-dateutil-2.8.2 python-ulid-2.2.0 six-1.16.0 sniffio-1.3.0 sqlite-fts4-1.0.3 sqlite-migrate-0.1b0 sqlite-utils-3.35.2 tabulate-0.9.0 tqdm-4.66.1 typing-extensions-4.8.0
(.venv) ➜  /tmp python3 -c 'import llm'
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/private/tmp/.venv/lib/python3.11/site-packages/llm/__init__.py", line 18, in <module>
    from .plugins import pm
  File "/private/tmp/.venv/lib/python3.11/site-packages/llm/plugins.py", line 37, in <module>
    mod = importlib.import_module(plugin)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/opt/homebrew/Cellar/[email protected]/3.11.6_1/Frameworks/Python.framework/Versions/3.11/lib/python3.11/importlib/__init__.py", line 126, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/private/tmp/.venv/lib/python3.11/site-packages/llm/default_plugins/openai_models.py", line 14, in <module>
    import requests
ModuleNotFoundError: No module named 'requests'

Speculation as to the cause

I took a look at the release history of openai and saw that they’d just released 1.0 today, whereas the prior stable release had been 0.28.1 from September 26th. This made me think that perhaps this project doesn’t “pin” the version of openai that it uses. I checked setup.py and that does indeed seem to be the case.

So I’m thinking that openai 0.28.1 did include requests in its dependency tree, and llm was implicitely relying on that fact, by importing requests in its code but not including the dependency in its direct dependencies, instead relying on openai to bring it in. Which it did — but it appears that now it does not.

Experiment

To test the above hypothesis, I manually uninstalled `openai` and then installed `openai==0.28.1`. Once I did that, `import llm` no longer raised any error:

(.venv) ➜  /tmp pip uninstall openai     
Found existing installation: openai 1.0.1
Uninstalling openai-1.0.1:
  Would remove:
    /private/tmp/.venv/bin/openai
    /private/tmp/.venv/lib/python3.11/site-packages/openai-1.0.1.dist-info/*
    /private/tmp/.venv/lib/python3.11/site-packages/openai/*
Proceed (Y/n)? y
  Successfully uninstalled openai-1.0.1
(.venv) ➜  /tmp pip install openai==0.28.1
Collecting openai==0.28.1
  Using cached openai-0.28.1-py3-none-any.whl.metadata (11 kB)
Collecting requests>=2.20 (from openai==0.28.1)
  Using cached requests-2.31.0-py3-none-any.whl.metadata (4.6 kB)
Requirement already satisfied: tqdm in ./.venv/lib/python3.11/site-packages (from openai==0.28.1) (4.66.1)
Collecting aiohttp (from openai==0.28.1)
  Using cached aiohttp-3.8.6-cp311-cp311-macosx_11_0_arm64.whl.metadata (7.7 kB)
Collecting charset-normalizer<4,>=2 (from requests>=2.20->openai==0.28.1)
  Using cached charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl.metadata (33 kB)
Requirement already satisfied: idna<4,>=2.5 in ./.venv/lib/python3.11/site-packages (from requests>=2.20->openai==0.28.1) (3.4)
Collecting urllib3<3,>=1.21.1 (from requests>=2.20->openai==0.28.1)
  Using cached urllib3-2.0.7-py3-none-any.whl.metadata (6.6 kB)
Requirement already satisfied: certifi>=2017.4.17 in ./.venv/lib/python3.11/site-packages (from requests>=2.20->openai==0.28.1) (2023.7.22)
Collecting attrs>=17.3.0 (from aiohttp->openai==0.28.1)
  Using cached attrs-23.1.0-py3-none-any.whl (61 kB)
Collecting multidict<7.0,>=4.5 (from aiohttp->openai==0.28.1)
  Using cached multidict-6.0.4-cp311-cp311-macosx_11_0_arm64.whl (29 kB)
Collecting async-timeout<5.0,>=4.0.0a3 (from aiohttp->openai==0.28.1)
  Using cached async_timeout-4.0.3-py3-none-any.whl.metadata (4.2 kB)
Collecting yarl<2.0,>=1.0 (from aiohttp->openai==0.28.1)
  Using cached yarl-1.9.2-cp311-cp311-macosx_11_0_arm64.whl (61 kB)
Collecting frozenlist>=1.1.1 (from aiohttp->openai==0.28.1)
  Using cached frozenlist-1.4.0-cp311-cp311-macosx_11_0_arm64.whl.metadata (5.2 kB)
Collecting aiosignal>=1.1.2 (from aiohttp->openai==0.28.1)
  Using cached aiosignal-1.3.1-py3-none-any.whl (7.6 kB)
Using cached openai-0.28.1-py3-none-any.whl (76 kB)
Using cached requests-2.31.0-py3-none-any.whl (62 kB)
Using cached aiohttp-3.8.6-cp311-cp311-macosx_11_0_arm64.whl (343 kB)
Using cached async_timeout-4.0.3-py3-none-any.whl (5.7 kB)
Using cached charset_normalizer-3.3.2-cp311-cp311-macosx_11_0_arm64.whl (118 kB)
Using cached frozenlist-1.4.0-cp311-cp311-macosx_11_0_arm64.whl (46 kB)
Using cached urllib3-2.0.7-py3-none-any.whl (124 kB)
Installing collected packages: urllib3, multidict, frozenlist, charset-normalizer, attrs, async-timeout, yarl, requests, aiosignal, aiohttp, openai
Successfully installed aiohttp-3.8.6 aiosignal-1.3.1 async-timeout-4.0.3 attrs-23.1.0 charset-normalizer-3.3.2 frozenlist-1.4.0 multidict-6.0.4 openai-0.28.1 requests-2.31.0 urllib3-2.0.7 yarl-1.9.2
(.venv) ➜  /tmp python3 -c 'import llm'
(.venv) ➜  /tmp echo $?
0 

HTH!