transformer icon indicating copy to clipboard operation
transformer copied to clipboard

Update to gotch 0.9.0

Open jdrake opened this issue 10 months ago • 2 comments

Would be great to update transformer to work with https://github.com/sugarme/gotch/tree/v0.9.0

For example in /transformer/util/activation.go, the MustGelu method signature changed to accept a string and a bool:

func (g GeluActivation) Fwd(x *ts.Tensor) (retVal *ts.Tensor) {
	return x.MustGelu("???", false)
                           ^^^
}

jdrake avatar Oct 20 '23 16:10 jdrake

Hitting the same issue, lets get this fixed

Chief-Alchemist avatar Oct 23 '23 09:10 Chief-Alchemist

func (g GeluActivation) Fwd(x *ts.Tensor) (retVal *ts.Tensor) {
	return x.MustGelu("none", false)
}

you can replace with "none" or "tanh" according to libtorch docs

aorticweb avatar Nov 11 '23 00:11 aorticweb