aphrodite-engine
aphrodite-engine copied to clipboard
[WIP] feat: T5 support
This PR adds support for the T5 family of models, a series of encoder-decoder models.
Currently a work in progress.
TODO:
- [x] Add the modeling code
- [x] Add T5Config
- [x] Modify attention to account for the encoder
- [x] Modify the cache engine
- [ ] Make the weight loading logic more robust
- [ ] Possibly support quantization
No progress on this? I want to run a prompt enhancer on aphrodite 24/7 and it will give those with low-powered PCs some new way to help :)