Mor Zusman

fixing issue https://github.com/microsoft/DeepSpeed/issues/2760 basically just fixing from_blob strides , before the fix the functions returned key and value cache that was populated by zeros.

[BUG] Zeros in returned k_v cache when get_present = True

1

**Describe the bug** Running a forward pass with `get_present = True` returns wrong values for key and value tensor. They contain a lot of zeros. **To Reproduce** Steps to reproduce...

[Model] Jamba support

3

Add Jamba support to vLLM, This PR comprises two parts: the Jamba modeling file and the Mamba memory handling. Since Jamba is a hybrid model (which alternates between mamba and...

[BugFix] get_and_reset only when scheduler outputs are not empty

2

Following #4115 , Thanks to https://github.com/vllm-project/vllm/pull/4115#discussion_r1670888574 , Noticed that `scheduler.finished_requests_ids` could reset without sending the finished requests ids back to the workers. This PR resets the `finished_requests_ids` only when the...

feat: Initial state support for Mamba SSM (1)

2

Add chunked prefill / use initial state capability to Mamba ssm ( Mamba 1 ) , Done it by prepending the last forward pass state to the FWD pass kernel...