Lesson 19 - Seq2Seq - Date translator - Solutions.ipynb: Padding Doubt

[gangooteli]

Why the input is forward padded? For e.g. <PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD><PAD>31, Aug 1986

Is there specific reason for that? What difference does it make to do padding before the input sequence instead of padding after input sequence. Please explain.

[lofar788]

My guess would be that you want as much data as possible when your sending the encoder state into the decoder. If you padded at the end of the data the LSTM might forget the stuff at the beginning.

[sachinruk]

^correct @lofar788 . I want all the most important data closer to the final output vector. So this ought to make it easier for the network. Technically speaking it ought to be able to (learn to) ignore the paddings by itself.

[gangooteli]

@sachinruk @lofar788 We can also use seq_len and pass sequence length such that dynamic_rnn will do learning upto length of each sequence. In this we don't even need to worry about learning from pad.

[jannisborn]

@gangooteli Interesting option, would you mind sharing your code?