Cade Daniel comments

Results 121 comments of


                                            Cade Daniel

[Speculative decoding 1/9] Optimized rejection sampler

cc @LiuXiaoxuanPKU @WoosukKwon @zhuohan123 @Yard1

[Speculative decoding 1/9] Optimized rejection sampler

The next PR will be https://github.com/cadedaniel/vllm-public/pull/1, will create it once this is merged.

[Misc]: Implement CPU/GPU swapping in BlockManagerV2

retrying

[Misc]: Implement CPU/GPU swapping in BlockManagerV2

retried

[Misc]: Implement CPU/GPU swapping in BlockManagerV2

thanks @Kaiyang-Chen . i was out last week. I will get this merged this week, thanks for all your work!

[Misc]: Implement CPU/GPU swapping in BlockManagerV2

Enabled auto merge

[Misc]: Implement CPU/GPU swapping in BlockManagerV2

Done! thanks again @Kaiyang-Chen

[RFC]: Automate Speculative Decoding

@brotherchen more info [here](https://github.com/vllm-project/vllm/issues/1742) on lookahead decoding in vLLM. currently not being worked on as far as I know.

[Core] Sliding window for block manager v2

@mmoskal can you check the merge conflict? will merge after

[Core] Sliding window for block manager v2

as soon are tests are green let's merge. cc @rkooo567 for next week.