sglang
sglang copied to clipboard
[Feature] Reorganize all the docs
Checklist
- [x] 1. If the issue you raised is not a feature but a question, please raise a discussion at https://github.com/sgl-project/sglang/discussions/new/choose Otherwise, it will be closed.
- [x] 2. Please use English, otherwise it will be closed.
Motivation
-
Quick Start: Sending Requests move to Getting Started
- print_highlight
if is_in_ci, use html. else directly print it.
- differentiate two streaming https://docs.sglang.ai/backend/send_request.html
- add description to this docs:
- https://docs.sglang.ai/backend/openai_api_completions.html add link to it
-
change meta-llama/Meta-Llama-3.1-8B-Instruct to
meta-llama/Llama-3.2-1B-Instruct
-
deepseek
This should link to deepseek official and saying like:
SGLang is recognized as one of the top engines for [DeepSeek model inference](deepseek official). Refer to[ installation and launch](https://github.com/sgl-project/sglang/tree/main/benchmark/deepseek_v3#installation--launch) to fire up DeepSeek V3/R1 on SGLang.
- https://docs.sglang.ai/backend/openai_api_vision.html
Add link to this https://github.com/sgl-project/sglang/blob/main/examples/runtime/engine/offline_batch_inference_vlm.py to tell users we can use engine to do generate.
- https://docs.sglang.ai/backend/offline_engine_api.html move this to
examples
and give links at the beginning.
- https://docs.sglang.ai/backend/function_calling.html remove this or give link to what is function calling
- https://docs.sglang.ai/frontend/frontend.html
We can review simon's PR.
- https://docs.sglang.ai/router/router.html
remove the $
- https://docs.sglang.ai/references/hyperparameter_tuning.html
Use router if you are not using the engine.
Tell them how to use the least VRAM.
Link to server args.
- https://docs.sglang.ai/references/accuracy_evaluation.html
Let's do it next time. Link to benchmark and add readme in benchmark.
- https://docs.sglang.ai/references/multi_node.html
- https://docs.sglang.ai/references/quantization.html
-
references: move some part to backend.
-
take AMD back:
https://docs.sglang.ai/references/amd_configure.html
Related resources
No response