'make build' error on chroma-hnswlib

[tuxxon]

Describe the bug

chroma-hnswlib got error when 'make build'

Setup and configuration

Current version:

 opendevin  git log -n 1
commit 6e3b554317de7bc5d96ef81b4097287e05c0c4d0 (HEAD -> main, origin/main, origin/HEAD)
Author: RaGe <[email protected]>
Date:   Sun Apr 7 15:57:31 2024 -0400

    Create a CommandExecutor abstract class (#874)
    
    * Create abstract CommandExecutor class
    
    * Use CommandExecutor for Sandbox

My config.toml and environment vars (be sure to redact API keys):

 OS: Apple M3 Max, Sonoma 14.4.1
 Python 3.12.2
 node.js 18.19.1
 poetry 1.8.2

My model and agent (you can see these settings in the UI):

• Model: n/a
• Agent: n/a

Commands I ran to install and run OpenDevin:

$ make build

Steps to Reproduce:

1. git clone ....
2. cd opendevin
3. make build

Logs, error messages, and screenshots:

Building project...
Checking dependencies...
Checking Python installation...
Python 3.12.2 is already installed.
Checking npm installation...
npm 10.2.4 is already installed.
Checking Docker installation...
Docker version 25.0.3, build 4debf41 is already installed.
Checking Poetry installation...
Poetry (version 1.8.2) is already installed.
Dependencies checked successfully.
Pulling Docker image...
Using default tag: latest
latest: Pulling from opendevin/sandbox
f4bb4e8dca02: Pull complete 
fdfff9a8a16c: Pull complete 
Digest: sha256:486e3da1d7c8ec54a2e3269071eed9b91bc75b82ca5d1ac1760d33883c39e575
Status: Downloaded newer image for ghcr.io/opendevin/sandbox:latest
ghcr.io/opendevin/sandbox:latest

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview ghcr.io/opendevin/sandbox
Docker image pulled successfully.
Installing Python dependencies...
/bin/sh: chroma-hnswlib: command not found
Installing ...
Creating virtualenv opendevin-kgBG19h1-py3.12 in /Users/tuxxon/Library/Caches/pypoetry/virtualenvs
Collecting chroma-hnswlib
  Downloading chroma-hnswlib-0.7.3.tar.gz (31 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "/Users/tuxxon/Library/Caches/pypoetry/virtualenvs/opendevin-kgBG19h1-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/tuxxon/Library/Caches/pypoetry/virtualenvs/opendevin-kgBG19h1-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/tuxxon/Library/Caches/pypoetry/virtualenvs/opendevin-kgBG19h1-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/pip-build-env-4avxzj0n/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/pip-build-env-4avxzj0n/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/pip-build-env-4avxzj0n/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 73, in <module>
        File "<string>", line 90, in BuildExt
      ValueError: list.remove(x): x not in list
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
make[1]: *** [install-python-dependencies] Error 1
make: *** [build] Error 2

Additional Context

[enyst]

Can you please try poetry update ?

[tuxxon]

$ poetry update
Updating dependencies
Resolving dependencies... (30.0s)

Package operations: 196 installs, 0 updates, 0 removals

  - Installing certifi (2024.2.2)
  ....
  - Installing chroma-hnswlib (0.7.3): Failed

  ChefBuildError

  Backend subprocess exited when trying to invoke get_requires_for_build_wheel
  
  Traceback (most recent call last):
    File "/Users/tuxxon/Library/Application Support/pypoetry/venv/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
      main()
    File "/Users/tuxxon/Library/Application Support/pypoetry/venv/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 335, in main
      json_out['return_val'] = hook(**hook_input['kwargs'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/Users/tuxxon/Library/Application Support/pypoetry/venv/lib/python3.12/site-packages/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
      return hook(config_settings)
             ^^^^^^^^^^^^^^^^^^^^^
    File "/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/tmp_hqf9m2b/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
      return self._get_build_requires(config_settings, requirements=['wheel'])
             ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
    File "/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/tmp_hqf9m2b/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
      self.run_setup()
    File "/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/tmp_hqf9m2b/.venv/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
      exec(code, locals())
    File "<string>", line 73, in <module>
    File "<string>", line 90, in BuildExt
  ValueError: list.remove(x): x not in list
  

  at ~/Library/Application Support/pypoetry/venv/lib/python3.12/site-packages/poetry/installation/chef.py:164 in _prepare
      160│ 
      161│                 error = ChefBuildError("\n\n".join(message_parts))
      162│ 
      163│             if error is not None:
    → 164│                 raise error from None
      165│ 
      166│             return path
      167│ 
      168│     def _prepare_sdist(self, archive: Path, destination: Path | None = None) -> Path:

Note: This error originates from the build backend, and is likely not a problem with poetry but with chroma-hnswlib (0.7.3) not supporting PEP 517 builds. You can verify this by running 'pip wheel --no-cache-dir --use-pep517 "chroma-hnswlib (==0.7.3)"'.

  - Installing contourpy (1.2.1)
 ...
  - Installing uvicorn (0.29.0)

Poetry also got error.

And chroma-hnswlib still got the same error.

[tuxxon]

Do I have to downgrade for poetry?

[tuxxon]

Could you provide Dockerfile?

[blue-hope]

I suffered from same issue, but resolved when ran poetry update and make build again

[SmartManoj]

Use Python 3.11 When should you upgrade to Python 3.12?

[tuxxon]

$ pyenv install 3.11.8
$ pyenv global 3.11.8
$ pyenv local 3.11.8
$ python -V
Python 3.11.8

$ make build
Building project...
Checking dependencies...
Checking Python installation...
Python 3.11.8 is already installed.
Checking npm installation...
npm 10.2.4 is already installed.
Checking Docker installation...
Docker version 25.0.3, build 4debf41 is already installed.
Checking Poetry installation...
Poetry (version 1.8.2) is already installed.
Dependencies checked successfully.
Pulling Docker image...
Using default tag: latest
latest: Pulling from opendevin/sandbox
Digest: sha256:486e3da1d7c8ec54a2e3269071eed9b91bc75b82ca5d1ac1760d33883c39e575
Status: Image is up to date for ghcr.io/opendevin/sandbox:latest
ghcr.io/opendevin/sandbox:latest

What's Next?
  View a summary of image vulnerabilities and recommendations → docker scout quickview ghcr.io/opendevin/sandbox
Docker image pulled successfully.
Installing Python dependencies...
/bin/sh: chroma-hnswlib: command not found
Installing ...
Collecting chroma-hnswlib
  Using cached chroma-hnswlib-0.7.3.tar.gz (31 kB)
  Installing build dependencies ... done
  Getting requirements to build wheel ... error
  error: subprocess-exited-with-error
  
  × Getting requirements to build wheel did not run successfully.
  │ exit code: 1
  ╰─> [19 lines of output]
      Traceback (most recent call last):
        File "/Users/tuxxon/Library/Caches/pypoetry/virtualenvs/opendevin-kgBG19h1-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 353, in <module>
          main()
        File "/Users/tuxxon/Library/Caches/pypoetry/virtualenvs/opendevin-kgBG19h1-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 335, in main
          json_out['return_val'] = hook(**hook_input['kwargs'])
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/Users/tuxxon/Library/Caches/pypoetry/virtualenvs/opendevin-kgBG19h1-py3.12/lib/python3.12/site-packages/pip/_vendor/pyproject_hooks/_in_process/_in_process.py", line 118, in get_requires_for_build_wheel
          return hook(config_settings)
                 ^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/pip-build-env-q_x5bx13/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 325, in get_requires_for_build_wheel
          return self._get_build_requires(config_settings, requirements=['wheel'])
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
        File "/private/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/pip-build-env-q_x5bx13/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 295, in _get_build_requires
          self.run_setup()
        File "/private/var/folders/fh/49mn2px92_36css62d60fxn00000gn/T/pip-build-env-q_x5bx13/overlay/lib/python3.12/site-packages/setuptools/build_meta.py", line 311, in run_setup
          exec(code, locals())
        File "<string>", line 73, in <module>
        File "<string>", line 90, in BuildExt
      ValueError: list.remove(x): x not in list
      [end of output]
  
  note: This error originates from a subprocess, and is likely not a problem with pip.
error: subprocess-exited-with-error

× Getting requirements to build wheel did not run successfully.
│ exit code: 1
╰─> See above for output.

note: This error originates from a subprocess, and is likely not a problem with pip.
make[1]: *** [install-python-dependencies] Error 1
make: *** [build] Error 2

Even though installed python 3.11.8, still got the same error.

[SmartManoj]

-py3.12

still uses 3.12

[yellowcandle]

same issue here. fixed by:
uninstall poetry first,
and switch your local python version to 3.11.7 by pyenv local 3.11.7.
then, reinstall poetry.

[tuxxon]

@SmartManoj @yellowcandle It works. Thank you so much.

[vineetdaniel]

I followed these steps and it worked for me: $ pyenv install 3.11.8 $ pyenv global 3.11.8 $ pyenv local 3.11.8 $ python -V

python version changed after restarting shell. source command didn't worked.

[Aboitier]

It worked for me after having installed poetry with Python 3.11.8 Once python -V returns 3.11.8 : pipx uninstall poetry pipx install poetry --python $(which python)