open-interpreter icon indicating copy to clipboard operation
open-interpreter copied to clipboard

Add llm_drop_params setting to avoid AnthropicException

Open nobu007 opened this issue 1 year ago • 4 comments

litellm.drop_params=True can avoid this error(https://github.com/OpenInterpreter/open-interpreter/issues/1240). Error in chat: AnthropicException - anthropic does not support parameters I think drop_params is good altanative for debugging or workaround of future errors.

Describe the changes you have made:

This option allow litellm.drop_params=True.

Reference any relevant issues (e.g. "Fixes #000"):

Related with https://github.com/OpenInterpreter/open-interpreter/issues/1240.

Pre-Submission Checklist (optional but appreciated):

  • [x] I have included relevant documentation updates (stored in /docs)
  • [x] I have read docs/CONTRIBUTING.md
  • [x] I have read docs/ROADMAP.md

I found "Use Anthropic function calling" in ROADMAP.md. Is #1240 just not implemaneted?

OS Tests (optional but appreciated):

  • [ ] Tested on Windows
  • [ ] Tested on MacOS
  • [x] Tested on Linux

Test(works fine)

interpreter --model claude-3-haiku-20240307 -y --llm_drop_params

Note1

This error is probably resolved by this PR.

Error in chat: AnthropicException - {"type":"error","error":{"type":"invalid_request_error","message":"messages: roles must alternate between \"user\" and \"assistant\", but found multiple \"user\" roles in a row"}}

Note2

"litellm.modify_params" helps activate here code in litellm. The first message is empty or non user, dummy message append in anthropic model. It is not important, but helps for me.

nobu007 avatar Apr 27 '24 03:04 nobu007

Hey @nobu007, thanks for testing this with Anthropic and making this fix!

Should we just set litellm.drop_params = True? It seems like that could never be bad— if LiteLLM doesn't want to pass a param into the LLM, I feel like we should just let it silently drop it. Any side effects of that that you can see?

KillianLucas avatar May 04 '24 22:05 KillianLucas

@KillianLucas Yes. In my for 2 weeks try, it seems no side effects forme. drop_params is used around here.

Btw, modify_params is also useful for Anthropic. modify_params is used around here and here.

How about adding this also?

nobu007 avatar May 12 '24 06:05 nobu007

@KillianLucas I added "llm_modify_params" also. Please check it!

nobu007 avatar Jun 02 '24 03:06 nobu007

@KillianLucas I rebased.

nobu007 avatar Jun 25 '24 00:06 nobu007