João Lages

@maciejkula yeah, that's it. Each time step `i` could have more than one item associated with it. This would be very useful in cases where 'the order of the items...

Ok, I'm starting to understand. I missed [this pad](https://github.com/maciejkula/spotlight/blob/fa3655996dccdd33b419cef86b4e95da5f1196c0/spotlight/sequence/representations.py#L219) that you do to your input. You mask the input so that the task is kind of like 'predicting the last...

While predicting, for me it would make sense if you would use the whole input, non-padded, and the whole output afterwards, not only a portion of it related with the...

Ah ok, I think I finally understood. Only `user_representations` will contain vectors that try to become as clone as the input vector, after passing through the embedding layer. This means...

It'd be cool if we could have a custom hidden layer with that behavior to add more non-linearities and transformations to the model

What's the big advantage over training only one time step at a time? By that means, each `i` would have a single bakprop

From what I understood, in the GPT-2 experiment, you only changed a [single Conv1D layer](https://github.com/microsoft/LoRA/blob/aa68d8a021c7ba08973e35fdfdc76338fdbfad57/examples/NLG/src/model.py#L95), right? That makes more sense in terms of training speeds.

Not really. One would need to look deeply into the code. Most of these models preprocess the SPIDER dataset previously and then use another script to train the model on...

Thanks for your reply. That is what I am doing. Nevertheless, it seems that while using 2 optimizers the loss lowers way faster than comparing with one optimizer. what might...

Nevermind, I had a typo, 2 optimizers vs 1 optimizers produces more or less the same it seems. Still having the loss problem though