FasterTransformer icon indicating copy to clipboard operation
FasterTransformer copied to clipboard

Published 20 hours ago • verified publisher icon NVIDIA

mt5 support？

Open Elisewyh opened this issue 2 years ago • 1 comments

Description

Is it possible to support mt5 acceleration by changing the activation function of mt5 to relu? Are there other things to pay attention to?

Reproduced Steps

1、I changed the activation function of mt5 from gated-gelu to relu, 
2、then finetuned my model, the evaluation index did not drop, 
3、and then accelerated with FT, the generated results became worse，
4、I want to ask if there are other things I didn't notice.

Elisewyh avatar Jul 14 '22 08:07 Elisewyh

Please refer https://github.com/NVIDIA/FasterTransformer/issues/225.

byshiue avatar Jul 14 '22 08:07 byshiue

mt5 is supported in latest release. You can refer https://github.com/NVIDIA/FasterTransformer/blob/main/docs/t5_guide.md#running-mt5.

byshiue avatar Aug 16 '22 03:08 byshiue

Close this bug because it is inactivated. Feel free to re-open this issue if you still have any problem.

byshiue avatar Sep 08 '22 07:09 byshiue