Open-Assistant icon indicating copy to clipboard operation
Open-Assistant copied to clipboard

Add initial load tests

Open jackapbutler opened this issue 2 years ago • 0 comments

  • closes #1622

I've used locust to write a basic load test which will hit two endpoints sequentially that mimic the text-client in the inference server.

  1. /chat to start a new conversation with chat_id
  2. /chat/{chat_id}/message to send a message to the Assistant

An isolated load test user workflow is summarised by first spawning $X$ users every $T$ seconds to a maximum amount of $N$ concurrent users

  1. A user starts a conversation with the Assistant
  2. Then they enter a conversation loop
    1. Send a chat message to the Assistant
    2. Wait until Assistant responds
    3. Wait $S$ further seconds
    4. Repeat

Initial results showed the Bot failing (potentially due to a race condition) and I'm currently investigating the source of that error. The main error happens inside of add_prompter_message() and is is caused by the chat.pending_message_request status being non None giving rise to a HTTPError of “Already pending”.

jackapbutler avatar Feb 16 '23 15:02 jackapbutler