Rain Jiang

Can we support other LLM mode?

12

Is it possible to integrate with gpt4all instead of openai?

Support to compile base on pytorch 2.1.0 - https://github.com/pytorch/pytorch/blob/main/torch/csrc/api/include/torch/all.h#L4

Is it possible to support monoio runtime?

1

[Monoio](https://github.com/bytedance/monoio/tree/master) is a thread-per-core Rust runtime with io_uring/epoll/kqueue. Is it possible to support monoio runtime at pyo3 ?

[Bug] PD disaggregating lead to AssertionError on sglang and nixl

1

### Checklist - [x] 1. I have searched related issues but cannot get the expected help. - [x] 2. The bug has not been fixed in the latest version. -...

Support Custom Backend via config, configuration example ``` chunk_size: 32 local_device: "cpu" remote_serde: "naive" enable_blending: False max_local_cpu_size: 20 custom_backend: "lmcache.dummy_backend.DummyBackend" # custom storage backend # Whether retrieve() is pipelined or...