Tom Dörr

Logprobs are almost the same for all choices

8

I'm retrieving the logprobs of ``` "Final Answer: " + sgl.gen("answer_bool", choices=['True', 'False']) ``` but they are almost exactly the same for many different inputs ``` logprobs: [-0.629454493522644, -0.6299992203712463] logprobs:...

Switch to non gated models

1

When running ``` python -m sglang.launch_server --model-path meta-llama/Llama-2-7b-chat-hf --port 30000 ``` I get ``` Cannot access gated repo for url https://huggingface.co/meta-llama/Llama-2-7b-chat-hf/resolve/main/config.json. Access to model meta-llama/Llama-2-7b-chat-hf is restricted. You must be...

Don't get API response when sending images

1

I loaded Llava v1.6 34B on my server ``` export DISABLE_NEST_ASYNCIO=True model=liuhaotian/llava-v1.6-34b tokenizer=liuhaotian/llava-v1.6-34b-tokenizer CUDA_VISIBLE_DEVICES=0,1 python3 -m sglang.launch_server --model-path $model --tokenizer-path $tokenizer --port 30813 --tp 2 ``` It works when I...

Add logprobs hfclientvllm

7

This is part of https://github.com/stanfordnlp/dspy/issues/879. I tested it and it behaves the same as the other PR. ``` lm = dspy.HFClientVLLM(model="NurtureAI/Meta-Llama-3-8B-Instruct-32k", port=38242, url="http://localhost", max_tokens=4) test_text = "This is a test...

Failed to parse JSON response: {"detail":"Not Found"}

6

The code im running: ``` lm = dspy.HFClientVLLM(model="NurtureAI/Meta-Llama-3-8B-Instruct-32k", port=38242, url="http://localhost", max_tokens=4) test_text = "This is a test article. abc" output_normal = lm(test_text) print("output_normal:", output_normal) ``` I start the server with:...

Does it make sense to remove the `dspy.` if a class is not actually accessible over it? In case that's wanted it might make sense to actually allow access with...

I'm using HFClientVLLM and set `num_threads=32` but the time it takes evaluate to finish goes up linear with the number of samples. This shouldn't be the case since vllm is...

Tab filename completion causes aider to hang

11

### Issue If I tab the filename gets filled out, but the name is surrounded by ` characters. This causes aider to hang and I need to start a new...

Import error

2

### Issue I updated aider in hope to fix my issue in https://github.com/paul-gauthier/aider/issues/995 However after updating Aider I can't start it anymore at all: ``` ❯ aider --sonnet --vim Traceback...