abd

You might want to try the vLLM library. I used that to deploy the Mistral-nemo model in a multi-node, multi-gpu setting. Reference: https://docs.mistral.ai/deployment/self-deployment/vllm/ I could be wrong, but I think...

mistral-finetune has a requirement of torch==2.2, whereas mistral-inference has a requirement of torch==2.3.0 for all but the first release. Is there anyway to have the two of them in the...