sglang
sglang copied to clipboard
sgl-project
Outlines integration
This package looks awesome! I was wondering why you decided to copy Outlines' code instead of importing the FSMs directly from outlines? There are several improvements on the performance of guided generation in the pipeline and you will be missing out on those. By importing you get better as we get better :)
@rlouf We actually tried to import rather than copy the code. We met some dependency issues before, but we are happy to resolve the issues. @hnyls2002 Could you put your error messages here?
Glad to work with you! I tried to import outlines a few weeks ago when your version was 0.0.22. The dependency error is
Could not apply nest_asyncio: Can't patch loop of type <class 'uvloop.Loop'>
This is because nest_asyncio is not compatible with our dependencies. When I looked into your repo today, I was delighted that you made nest_asyncio an optional dependency.
https://github.com/outlines-dev/outlines/blob/65ed0f7e9f900701ba62479982ee29bb47cc2738/outlines/base.py#L17-L25
Great to see this! I will import your library into ours in a short time.
By the way, we just implemented simple regex for constraint decoding; later, we also want to support more complex constraint decoding, such as CFGs. I am happy to see you outlines did a great job in this field!
@rlouf Hi, I have made outlines an import in this PR(#60).
But when I set vllm>=0.2.5 and outlines>=0.0.24 as dependencies at the same time， we meet a conflict error:
ERROR: Cannot install sglang, sglang[srt]==0.1.5 and vllm==0.2.5 because these package versions have conflicting dependencies.
The conflict is caused by:
sglang[srt] 0.1.5 depends on pydantic; extra == "srt"
outlines 0.0.24 depends on pydantic>=2.0
vllm 0.2.5 depends on pydantic==1.10.13
sglang[srt] 0.1.5 depends on pydantic; extra == "srt"
outlines 0.0.24 depends on pydantic>=2.0
vllm 0.2.7 depends on pydantic==1.10.13
sglang[srt] 0.1.5 depends on pydantic; extra == "srt"
outlines 0.0.24 depends on pydantic>=2.0
vllm 0.2.6 depends on pydantic==1.10.13
Do you have any idea how to solve this?
We will likely need to wait until https://github.com/vllm-project/vllm/pull/2468 is merged on the vLLM side, which will relax the pydantic version.
It should soon be ok, vLLM merged https://github.com/vllm-project/vllm/pull/2531 that drops dependency on Pydantic v1.
vLLM released 0.3.0 which bumps the Pydantic version to >= 2.0. We'll cut a new release in Outlines today or tomorrow.
@rlouf Hi, I have imported outlines as a dependency(#168). Thanks for your great work on FSM and regex!