neolithic5452 issues

Repositories
Issues
Comments

Results 3 issues of


                                            neolithic5452

AMX only? re: KTransformers+SGLang Inference Deployment

Hi is the KTransformers+SGLang Inference Deployment only supported for AMX CPUs? Or can we try this with Epyc as well?

Fix broken TensorRT-LLM link to deepseekv3

Fix broken TensorRT-LLM link to deepseekv3 examples

Bug: Inference hangs when kv cache gets full

### What happened? I was running the Aider Polyglot against Kimi K2 Thinking ubergarm smol_iq3_ks. Excellent results by the way!!! Anyways KV cache got full every now and again I...