VQA-Keras-Visual-Question-Answering icon indicating copy to clipboard operation
VQA-Keras-Visual-Question-Answering copied to clipboard

Here you perform Hadamard product/ point-wise product b/w image & word layers. How to perform Bilinear product?

Open WrathofBhuvan11 opened this issue 4 years ago • 0 comments

let's say there are 2 encoders for image & word namely e_1 & e_2 respectively. we know that both e_1 and e_2 has shape of (None, 1024). and performing Hadamard/ point wise multiplication results in output tensor of shape (None, 1024) and this is connected to Linear DNN & so on . My question is how to implement a keras layer which performs Bilinear tensor product b/w e_1 & e_2. output = e_1 ⊗ e_2 so shape of output is (None, 1024, 1024) which can be connected to CNN down the model. But how to implement This Keras layer for Bilinear product, e_1 ⊗ e_2 which takes (None, 1024) shaped input and gives out output of shape (none, 1024, 1024) ??

WrathofBhuvan11 avatar Sep 21 '20 10:09 WrathofBhuvan11