[Feature/Integration] Add Integration Path with vLLM Semantic Router

[sadath-12]

🚀 Feature Description and Motivation

can semantic router ( https://github.com/vllm-project/semantic-router ) be integrated into aibrix's routing component ?

Use Case

This can give more intelligence to aibrix's routing

Proposed Solution

No response

[googs1025]

I am very happy to see this issue track, and I am looking forward to this feature being integrated into aibrix. 😄

[sadath-12]

@googs1025 can I pick this one and see where it goes ?

[googs1025]

Regarding routers, Aibrix currently uses Envoy Gateway for integration. The source code is here: https://github.com/vllm-project/aibrix/tree/main/pkg/plugins/gateway

If you have any ideas, please provide docs or share them in this issues. Other maintainers will actively provide answers and responses! cc @Jeffwan @varungup90

[sadath-12]

cool , let me take it from there @googs1025

[Jeffwan]

/cc @Xunzhuo please also help evaluate it

[Xunzhuo]

@Jeffwan Yes, I think this should be easy to integrate.

[Xunzhuo]

Some work-in-progress integration:

• Dynamo: https://vllm-semantic-router.com/docs/proposals/nvidia-dynamo-integration
• PS: https://github.com/vllm-project/semantic-router/issues/295

If AIBrix is interested in, that would be cool, then I can help with the design and move this forward

[sadath-12]

Better to unassign me from this now to unblock this issue , I got some other stuffs in middle and not sure how much time ill need to come back to this

[Jeffwan]

@sadath-12 sounds good. We will ask @Xunzhuo 's help on this issue