Teach Machine to Comprehend Text and Answer Question with Tensorflow - Part I · Han Xiao Tech Blog

[hanxiao]

https://hanxiao.github.io/2018/04/21/Teach-Machine-to-Comprehend-Text-and-Answer-Question-with-Tensorflow/

[nehaboob]

Very nice blog. Looking forward to the next part.

Just a question - When we encode question or passage, are we keeping all the outputs(of Bi Directional GRU RNN) from whole passage and then adding attention over it Or just keeping last output of Bi Directional GRU RNN (This would be king of passage encoding) Below line from "Embedding and Encoding Layers" is not clear. q_encodes, output_dim = tf.concat(q_encodes, 2), 2 * hidden_size

[hanxiao]

All outputs from the encoder, i.e. if your sequence has length L, batch size is B, LSTM hidden units is D. Then q_encodes should be [B, L, D], or [B, L, 2*D] when using bi-directional LSTM.

The line you are pointing out is a typo, thanks and I will fix it soon!

[nehaboob]

Got it. Thanks.

[iamshang1]

This is great! This post really clarifies the core components of question answering systems. Please post part two soon!

[duythvn]

Thanks for sharing this. please keep posting .

[cbqin]

Great blog, when is the part two? thanks.

[casakan]

Good blog. Now that we tasted the dish, I want more......when you intend to publish part 2?

[hanxiao]

Hey guys, part II of this series is available now!

[Raman-Raje]

Hi... i was trying to implement your code for the sake of understanding... I am stuck at start_label used in the loss function.

start_loss = tf.losses.sparse_softmax_cross_entropy(labels=start_label, logit=start_logit)

I am new to this area. Can you please explain what I should put there..

[iamshang1]

start labels are the integers representing the positions at which the answer span starts for your batch