litellm icon indicating copy to clipboard operation
litellm copied to clipboard

[Feature]: Multiple calls to mock completion return different mock responses

Open alckasoc opened this issue 6 months ago • 0 comments

The Feature

https://docs.litellm.ai/docs/completion/mock_requests

Kind of like LangChain's FakeListChatModel.

llm = FakeListChatModel(
  responses=[
    "response1",
    "response2",
    ...  
  ]
)

Would it be possible for the completion function to have a class-based alternative that can have a list of mock responses?

Motivation, pitch

In my unit tests, I have a mock LLM that is called multiple times. I want a convenient way for this mock LLM to return different responses depending on the i-th call.

Thanks!

alckasoc avatar Aug 05 '24 09:08 alckasoc