[Feature] Refactor inherent LLM to LLM router

[BRama10]

This document outlines the steps in the AIOS roadmap for refactoring the LLM structure. Feel free to discuss any of the following topics, and add any other topics you'd like to talk about in this issue.

Kernel

• [ ] Multiple-core LLM support (one core means one LLM instance)
• [ ] Base LLM router with customized priority
  • [ ] Cost-first Router, focus on token price (close-sourced llm)
  • [ ] Efficiency-first Router, focus on inference speed (open-sourced llm)

Agents

• [ ] Connect agent workflow to router to allow for automatic agent creation

[Firy01]

A basic router with customized features gives the flexibility and eventually it will be going to be a specific router. Feels this is a step one and two for a specific router. One doesn't have any capability. Two is specific capabilities. Both eventually will be specific routers.