Alex Cheema comments

Repositories
Issues
Comments

Results 413 comments of


                                            Alex Cheema

Add speculative decoding support with draft models

## Code Review — PR #1188: Add speculative decoding support with draft models **CI status**: All checks passing (typecheck, Build and check on aarch64-darwin, x86_64-linux, aarch64-linux) ### Overview This PR...

Fix: Pass missing KV cache constants to generation loop (#1036)

On second thought, we don't actually want `RotatingKVCache` to be the default, in which case we should set these constants to `None`. I would like to see some numbers on...

Fix: Pass missing KV cache constants to generation loop (#1036)

Thank you for your contribution! The exo codebase has undergone significant architectural changes since this PR was opened (event sourcing, new placement system, runner rewrite), and this PR now has...