added support for vLLM server and docs for local models

[arnavsinghvi11]

-updated DSPy init with vLLM support -added vLLM to hf_client -added docs for local model testing with HFModel, TGI, vLLM, and MLC -added model initialization validation for HFClientTGI within latest changes

[andreapiso]

Any news on the state of this PR? Support for vLLM would be super.

[meditans]

I think this is especially misleading because VLLM support is already in the documentation, but the relevant HFClientVLLM class is only defined in this unmerged PR

[okhat]

This can now be closed but we'll need a new PR with just these bits:

        self._verify_model_with_server()
        self.kwargs = {
            "temperature": 0.1,
            "max_tokens": 75,
	@@ -42,6 +42,17 @@ def __init__(self, model, port, url="http://future-hgx-1", **kwargs):
            **kwargs,
        }

    def _verify_model_with_server(self):
        try:
            response = requests.get(f"{self.url}/info")
            response_json = response.json()
            server_model = response_json.get('model_id', '')
            if server_model != self.model:
                raise ValueError(f"Model mismatch. Client model: {self.model}, Server model: {server_model}")
        except Exception as e:
            print(f"Failed to verify model with server: {e}")
            raise

cc: @arnavsinghvi11

[chris-aeviator]

I'm happily using vllm via

import dspy lm = dspy.HFClientVLLM(model="mistralai/Mistral-7B-Instruct-v0.2", port=8000, url="http://192.168.88.192") dspy.settings.configure(lm=lm)