lightseq
lightseq copied to clipboard
mt5
any plan or example for mt5 support?
Not yet, we will evaluate the priority of t5/mt5. Could you tell us about your usage? e.g. pretrain/finetune/inference
Not yet, we will evaluate the priority of t5/mt5. Could you tell us about your usage? e.g. pretrain/finetune/inference
inference,I have trained my own generate model, and I test load model at server backend / tf-serving, they all too slow, so I think the sample/topk/topp strategy move to gpu may faster
support on https://github.com/bytedance/lightseq/tree/mt5_support. (fp32 only)