TensorFlow-Examples
TensorFlow-Examples copied to clipboard
Questions about biLSTM example
https://github.com/aymericdamien/TensorFlow-Examples/blob/9e1bb504f5d0f209d000997ce2ad95bb891798ab/examples/3_NeuralNetworks/bidirectional_rnn.py#L81
I think "outputs [-1]" and "outputs [0]" are equivalent (reversed) in this line of code, but the former (89%) works better than the latter (86%). Why?
outputs is a length T(=28) list of outputs (one for each input), which are concatenated forward and backward outputs.
output[-1] = concatenate(output_fw[-1], output_bw[0])
output[0] = concatenate(output_fw[0], output_bw[-1])
while:
output_fw[-1] not equal to output_bw[-1]
output_fw[0] not equal to output_bw[0]
Using tf.nn.bidirectional_dynamic_rnn
and concatenating its outputs outputs = tf.concat([outputs[0], outputs[1]], 2)
should perform better. It will create the graph on-the-fly and allow for variable batch sizes feeding.