Transformer_Relative_Position_PyTorch icon indicating copy to clipboard operation
Transformer_Relative_Position_PyTorch copied to clipboard

Question about the choice of `contiguous` usage

Open shabie opened this issue 2 years ago • 1 comments

Hi,

Thanks a lot for putting this implementation out there. It helped me a lot!

I was just curious why the contiguous() was called after permute or transpose but always before view. Would it be bad to call it after view?

Here is an example:

https://github.com/evelinehong/Transformer_Relative_Position_PyTorch/blob/fd1163eb72d74538932f4f2d62e9c2e876f232e5/relative_position.py#L79

shabie avatar Sep 16 '21 21:09 shabie

I believe certain operations like view does not behave as it should if the tensor is not contiguous, therefore the contigious() call.

oxjohanndiep avatar Aug 18 '22 07:08 oxjohanndiep