hugoWR

I'm my own experiments, it appears that using frozen sine-cosine positional embedding speed-up learning quite significantly. I guess it makes sense because that's one thing that the network doesn't have...