liquidpeachy

I configured relay and the haproxy to load balance streams. However, when I load up the streams in VLC or web browser, round robin doesn't happen. All streams stay on...

Running into the same error on the 13b and 70b chat models. Using a h100 80GB card. The 7b chat model works fine. Command (13b): `torchrun --nproc_per_node 2 example_chat_completion.py --ckpt_dir...