lightllm
lightllm copied to clipboard
LightLLM is a Python-based LLM (Large Language Model) inference and serving framework, notable for its lightweight design, easy scalability, and high-speed performance.
Results
125
lightllm issues
Sort by
recently updated
recently updated
newest added
The $\text{Pre}^3$ branch (https://github.com/ModelTC/LightLLM/tree/pre3-integrated) requires package `lightllm_constraint_decode_kernel` to run. However, I cannot find this package from the repositories in this organization. Is there any open-source implementation of $\text{Pre}^3$ that is...