Jack Butler

Implement basic load tests for inference-server

1

Implement basic tests for starting conversations and sending messages to the inference server through the /chat and /chat/{chat_id}/message endpoints

* closes #1622 I've used locust to write a basic load test which will hit two endpoints sequentially that mimic the `text-client` in the inference server. 1. `/chat` to start...

* closes #1622 I’ve used locust to write a basic load test which will hit two endpoints sequentially that mimic the text-client in the inference server. * `/chat` to start...

Add Late Prompt Tuning

1

# Overview [Late Prompt Tuning: A Late Prompt Could Be Better Than Many Prompts](https://arxiv.org/pdf/2210.11292.pdf) has shown that prompt tuning can sometimes be more effective if the prompt generation is placed...