Transformers4Rec icon indicating copy to clipboard operation
Transformers4Rec copied to clipboard

Add option for projecting the `TransformerBlock` output + specifying the activation function `tf_out_act`

Open sararb opened this issue 3 years ago • 0 comments

  • Add the option to project the output of the TransformerBlock pos_emb_pred with specific activation function tf_out_act:

pos_emb_pred =tf_out_act(nn.Linear(d_model, transformer_output_projection_dim)(pos_emb_pred)

  • This option is needed when the user want to project the output of the Transformer to specific dimension. For example to item_id embedding when using weight tying.

  • This projection will be included in the body of SequentialPredictionTask.

  • The projection_flag, the tf_out_act and the transformer_output_projection_dim should be set in the init of SequentialPredictionTask

sararb avatar Aug 11 '21 20:08 sararb