sglang

sglang copied to clipboard

As you are probably aware, huggingface-hub allows users to upload model & their implementation as a model repo and users can use them via AutoModel and sorts. For example: https://huggingface.co/microsoft/Florence-2-large/tree/main

This is great, because model uploaders don't have to wait for PR to get merged to share the model to the public compatible with huggingface ecosystem, and I think similar things can be done for sglang.

IMO this is such a great tool that is very underappreciated that can really shine from such ecosystem