worker-vllm
worker-vllm copied to clipboard
Best way to record data
I am looking to record input and output to the vLLM. I could put an HTTP proxy in front and capture the traffic, or modify your handler.
Rather than make changes to the code, I was wondering if you might have a better way to do this?