aphrodite-engine icon indicating copy to clipboard operation
aphrodite-engine copied to clipboard

[WIP] feat: T5 support

Open AlpinDale opened this issue 1 year ago • 1 comments

This PR adds support for the T5 family of models, a series of encoder-decoder models.

Currently a work in progress.

TODO:

  • [x] Add the modeling code
  • [x] Add T5Config
  • [x] Modify attention to account for the encoder
  • [x] Modify the cache engine
  • [ ] Make the weight loading logic more robust
  • [ ] Possibly support quantization

AlpinDale avatar Feb 07 '24 03:02 AlpinDale

No progress on this? I want to run a prompt enhancer on aphrodite 24/7 and it will give those with low-powered PCs some new way to help :)

db0 avatar May 11 '24 16:05 db0