transformer
transformer copied to clipboard
Update to gotch 0.9.0
Would be great to update transformer to work with https://github.com/sugarme/gotch/tree/v0.9.0
For example in /transformer/util/activation.go
, the MustGelu method signature changed to accept a string and a bool:
func (g GeluActivation) Fwd(x *ts.Tensor) (retVal *ts.Tensor) {
return x.MustGelu("???", false)
^^^
}
Hitting the same issue, lets get this fixed
func (g GeluActivation) Fwd(x *ts.Tensor) (retVal *ts.Tensor) {
return x.MustGelu("none", false)
}
you can replace with "none" or "tanh" according to libtorch docs