metaformer icon indicating copy to clipboard operation
metaformer copied to clipboard

metaformer has no positional encoding?

Open bio-mlhui opened this issue 8 months ago • 1 comments

I notice that Metaformer has no positional encoding(PE) either in the attention layers or at the model input, does this affect the performance? Is the positional encoding not necessary? What if metaformer is equipped with 2D sin-cos/learned PE?

bio-mlhui avatar Jun 15 '24 07:06 bio-mlhui