Yi Huang comments

Repositories
Issues
Comments

Results 2 comments of


                                            Yi Huang

feat: Store message db

I noticed the plain SQL statements embedded in the read/write methods and was wondering if the use of an ORM was ever discussed

[Feature Request] Add message queue between model clients and vllm or ollama servers

Could you please provide more context about the feature and its requirements? I have the following questions: 1. What is the size of the messages (average or maximum, or estimation)?...