TensorFlow-Examples icon indicating copy to clipboard operation
TensorFlow-Examples copied to clipboard

Questions about biLSTM example

Open aitsc opened this issue 6 years ago • 2 comments

https://github.com/aymericdamien/TensorFlow-Examples/blob/9e1bb504f5d0f209d000997ce2ad95bb891798ab/examples/3_NeuralNetworks/bidirectional_rnn.py#L81

I think "outputs [-1]" and "outputs [0]" are equivalent (reversed) in this line of code, but the former (89%) works better than the latter (86%). Why?

aitsc avatar Dec 03 '18 14:12 aitsc

outputs is a length T(=28) list of outputs (one for each input), which are concatenated forward and backward outputs. output[-1] = concatenate(output_fw[-1], output_bw[0]) output[0] = concatenate(output_fw[0], output_bw[-1]) while: output_fw[-1] not equal to output_bw[-1] output_fw[0] not equal to output_bw[0]

songt96 avatar Jan 14 '19 09:01 songt96

Using tf.nn.bidirectional_dynamic_rnn and concatenating its outputs outputs = tf.concat([outputs[0], outputs[1]], 2) should perform better. It will create the graph on-the-fly and allow for variable batch sizes feeding.

gugarosa avatar Feb 15 '19 17:02 gugarosa