attention weights should be reordered as well as the outputs in case inputs were sorted and packed when doing batch inference

[levhaikin]

https://github.com/huggingface/torchMoji/blob/198f7d4e0711a7d3cd01968812af0121c54477f8/torchmoji/model_def.py#L243

during inference in batches, if reorder_output becomes True (if input_seqs are not of PackedSequence), then outputs are correctly reordered to match input order.

however, if return_attention is requested (set to True) then it is (and that's the bug) returned in an order that does not match the inputs.

a possible fix could be in the following section in the code: https://github.com/huggingface/torchMoji/blob/198f7d4e0711a7d3cd01968812af0121c54477f8/torchmoji/model_def.py#L243 similarly to how outputs are reordered - we can add the attention reordering code under that same if statement:

reordered_for_weights = Variable(att_weights.data.new(att_weights.size()))
reordered_for_weights[perm_idx] = att_weights
att_weights = reordered_for_weights

does that make sense? am i missing anything?

[n0obcoder]

i added

reordered_for_weights = Variable(att_weights.data.new(att_weights.size())) reordered_for_weights[perm_idx] = att_weights att_weights = reordered_for_weights

after

if reorder_output:

but it doesnt seem to make any change in the ordering of the attention weights.

@levhaikin can you please help me with this, is case you have already solved this bug ?

Thanks in advance !