ZhangHaoQ comments

Repositories
Issues
Comments

Results 1 comments of


                                            ZhangHaoQ

v0.4.1: fatal error: concurrent map writes in prefix cache eviction

@gangmuk Hi, thanks for the follow-up. Here are the details: Load (RPS): around 20 QPS (≈20 requests/sec) LLM model: nemo-12b GPU model: NVIDIA 5090 The crash usually happens under this...