Transformers4Rec

Transformers4Rec copied to clipboard

- Add the option to project the output of the TransformerBlock `pos_emb_pred` with specific activation function `tf_out_act`: `pos_emb_pred =tf_out_act(nn.Linear(d_model, transformer_output_projection_dim)(pos_emb_pred)` - This option is needed when the user want to...

sararb

- Set the `attn_type` of the Transformer class (XLNet and Transformer-XL) based on the type of masking task - Should be included in _TransformerBlock code

sararb

Support to incremental training

1

Extending the embedding tables of categorical features for new values seen on incremental training. P.s. requires incremental preprocessing ( https://github.com/NVIDIA/NVTabular/issues/798 )

gabrielspmoreira

Currently blocked by #51.

marcromeyn

This ticket includes the 3 basic aggregators we support for non-sequential data: - [ ] ConcatFeatures - [ ] StackFeatures - [ ] ElementwiseSum

marcromeyn

Check if any required common session-based recommendation op is missing for CPU

gabrielspmoreira

Change nvtabular.io import to merlin.io.dataset

2

importing nvtabular.io is deprecated and causes crashes. `nvtabular.io` redirects to `merlin.io` for backwards compatibility, but `merlin.io` no longer contains `Dataset`. Instead, it should be `merlin.io.dataset.Dataset`. Fixes #484

samhedin

[Task] Import Dataset class from merlin.io instead of nvtabular.io

5

### Description Currently `Dataset` class is imported from `nvtabular.io` as in here https://github.com/NVIDIA-Merlin/Transformers4Rec/blob/main/merlin_standard_lib/utils/misc_utils.py#L199, but we should change this to `from merlin.io import Dataset`.

rnyak

### Description Support multi-GPU training for any prediction tasks provided by T4Rec. This can be done in two different ways. This task will potentially solve a customer issue: #423 ###...

sararb

[BUG] Loss drops to 0 after a few thousand steps when using fp16=True

7

The model training loss is suddenly dropping to 0 after over 1000 steps. I've tried iterating over different dataset as well but got the same behaviour. ## Details I am...

silpara