Travis J

GELU does not appear to support approximate tanh

4

The optional algorithm for GELU is to internally use tanh See more here: https://pytorch.org/docs/stable/generated/torch.nn.GELU.html#torch.nn.GELU I was expecting this to just work: var gelu = nn.GELU(approximate: "tanh"); When the approximate argument...