DragonFive

@JianyuZhan Hi, I try use your repo to test, I clone the code with main branch , and run ```bash pip install -e "python[all]" pip install flashinfer -i https://flashinfer.ai/whl/cu118/torch2.4/ ```...

> @DragonFive , Add "LLM" and "SamplingParams" in `sglang/__init__.py::__all__`, it is now imported in this **init**.py but not exposed, or you can try `from sglang.api import LLM, SamplingParams`. The code...

> [JianyuZhan#1](https://github.com/JianyuZhan/sglang/pull/1) — @JianyuZhan this compiles / addresses the typo Change all the 'model_overide_args' to 'model_override_args' in the repo will work well.

@JianyuZhan It runs well before I ungrade sglang to v0.3.0 on llama3.1-8b, after that I encounter some confused error ： ```bash 10:46:19.553 [10:46:19 TP0] Exception in ControllerSingle: 10:46:19.553 Traceback (most...

@benchislett I want to know whether it is possible to run the R1 model with pp=2 and tp=8, while running this draft model with tp=8. Because I have no 8xH200...

@fyabc Hi, can this patch support mutiple images in one prompt like follows: ``` Compute the value of the expression in the image below \nby using the emoji equations in...

We encounter this error too! looking forward to an official reply.