Deepseek-Chat or Deepseek-Coder

[bamit99]

I am trying to use Deepseek API from the dropdown list within OpenDevin. I chose Deepseek-Coder, provide the API key but it fails. I tried manually adding Deepseep\ in front of model too but it failed. Same for Deepseek chat. What is the correct format for providing Deepseek Model name within the console.

06:40:54 - opendevin:ERROR: llm.py:164 - LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=deepseek/deepseek-chat Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers. Attempt #5 | You can customize these settings in the configuration. 06:40:54 - opendevin:ERROR: agent_controller.py:147 - Error in loop Traceback (most recent call last): File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 698, in completion model, custom_llm_provider, dynamic_api_key, api_base = get_llm_provider( ^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 6400, in get_llm_provider raise e File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 6387, in get_llm_provider raise litellm.exceptions.BadRequestError( # type: ignore litellm.exceptions.BadRequestError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=deepseek/deepseek-chat Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers

During handling of the above exception, another exception occurred:

Traceback (most recent call last): File "/app/opendevin/controller/agent_controller.py", line 142, in _run finished = await self.step(i) ^^^^^^^^^^^^^^^^^^ File "/app/opendevin/controller/agent_controller.py", line 256, in step action = self.agent.step(self.state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/agenthub/codeact_agent/codeact_agent.py", line 223, in step response = self.llm.completion( ^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 330, in wrapped_f return self(f, *args, **kw) ^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 467, in call do = self.iter(retry_state=retry_state) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 368, in iter result = action(retry_state) ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 410, in exc_check raise retry_exc.reraise() ^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 183, in reraise raise self.last_attempt.result() ^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 449, in result return self.__get_result() ^^^^^^^^^^^^^^^^^^^ File "/usr/local/lib/python3.12/concurrent/futures/_base.py", line 401, in __get_result raise self._exception File "/app/.venv/lib/python3.12/site-packages/tenacity/init.py", line 470, in call result = fn(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^ File "/app/opendevin/llm/llm.py", line 188, in wrapper resp = completion_unwrapped(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 3222, in wrapper raise e File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 3116, in wrapper result = original_function(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/main.py", line 2226, in completion raise exception_type( ^^^^^^^^^^^^^^^ File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 9233, in exception_type raise e File "/app/.venv/lib/python3.12/site-packages/litellm/utils.py", line 9201, in exception_type raise APIConnectionError( litellm.exceptions.APIConnectionError: LLM Provider NOT provided. Pass in the LLM provider you are trying to call. You passed model=deepseek/deepseek-chat Pass model as E.g. For 'Huggingface' inference endpoints pass in completion(model='huggingface/starcoder',..) Learn more: https://docs.litellm.ai/docs/providers 06:40:54 - opendevin:INFO: agent_controller.py:190 - Setting agent(CodeActAgent) state from AgentState.RUNNING to AgentState.ERROR

[xingyaoww]

This is the config I used for deepseek chat before - can you try it?

[llm]
model = "openai/deepseek-chat"
base_url = "https://api.deepseek.com"
api_key = "YOUR KEY HERE"
temperature = 0.0

[bamit99]

I have tried both .env file and below setting within UI but same error. Where exactly are you making the changes especially about the Base URL?

[xingyaoww]

You should put in openai/deepseek-chat - openai tells litellm that you are using an OpenAI-compatible endpoint

[bamit99]

Unfortunately, Same!!

[xingyaoww]

Probably because we didn't pass api_base correctly from the front-end? Let me ask folks in frontend channel to see if it is true!

[xingyaoww]

Oh actually, can you try following this? Are you running this with docker?

https://opendevin.github.io/OpenDevin/modules/usage/llms/localLLMs#docker

--add-host host.docker.internal:host-gateway \
-e LLM_API_KEY="ollama" \
-e LLM_BASE_URL="http://host.docker.internal:11434" \

You can set LLM_BASE_URL if you are using docker to spin-up this

[bamit99]

It worked. Instead of Ollama, I provided the base URL of Deepseek. Again, while working with the Docker, we should be able to provide the Base URL of the API Service being used. I am not a developer so assuming that LiteLLM automatically adds it when specying the Model in UI like deepseek/deepseek-chat etc. and can use the Base URL automatically but somehow it was not working. With the -e switch, this worked. Thanks a ton for your help!! :)

export WORKSPACE_BASE=$(pwd)/workspace;

docker run
-it
--pull=always
-e LLM_API_KEY
-e SANDBOX_USER_ID=$(id -u)
-e WORKSPACE_MOUNT_PATH=$WORKSPACE_BASE
-e LLM_API_KEY="sk-XXXXXXXXXXXXXXXXXXXXXXXXXX"
-e LLM_BASE_URL="https://api.deepseek.com/v1"
-e DEFAULT_MODEL="deepseek-chat"
-v $WORKSPACE_BASE:/opt/workspace_base
-v /var/run/docker.sock:/var/run/docker.sock
-p 3000:3000
--add-host host.docker.internal:host-gateway
ghcr.io/opendevin/opendevin:0.5