vllm-project
a detailed technical report on architecture
Can you provide a detailed technical report on architecture, especially regarding the implementation architecture design of routing strategy, KV cache offloading, and autoscaling? In fact, it is still a bit difficult to clarify the responsibilities and relationships of current components.
Additionally, I'm also confused about the version correspondence between Aibrix and vLLM. The latest version of vLLM already supports PD disaggregation, but since Aibrix's underlying layer is based on the vLLM engine, why doesn't it support this yet? I suppose this confusion largely stems from a lack of understanding of the overall architecture.
For deployment engineers using Aibrix, understanding its architecture will likely facilitate better utilization of the platform.
@libin817927 P/D disaggregation involves many components like orchestration, proxy routing, connector relied components etc. We plan to provide an easy to use solutions and that's why we didn't say it's fully ready.
BTW, do you have any example to share? do you run 1p1d or xpyd?
for the tech report, did you get chance to look at
- https://arxiv.org/html/2504.03648v1
- https://aibrix.readthedocs.io/latest/
We'd love to polish the contents that you feel not clear.
