About the implementation of Poly Encoder

[Hannibal046]

Hi @chijames, thanks so much for this wonderful project! After digging into the code, I have two questions:

• Is there any special reason why masking is not implemented in this section? https://github.com/chijames/Poly-Encoder/blob/e5299e319c73666485667e8277d8ff0e2b7e253e/encoder.py#L72-L78

• Can we speed up the construction of poly_code_embeddings by using nn.Parameters? In this way, we don't need to create poly_ids and move it to GPU in every batches.

Thanks for your reply!