PyTorch Position Embedding

Install

pip install torch-position-embedding

Usage

from torch_position_embedding import PositionEmbedding

PositionEmbedding(num_embeddings=5, embedding_dim=10, mode=PositionEmbedding.MODE_ADD)

Modes:

• MODE_EXPAND: negative indices could be used to represent relative positions.
• MODE_ADD: add position embedding to the original tensor.
• MODE_CAT: concatenate position embedding to the original tensor.