CNN+LSTM (LRCN)

[muzixiang]

Hi, @anayebi , I am a newbie in keras, I want to do sequence classification (e.g., 2 classes, each element in the sequence is a 2d matrix or image) just like the first architecture in fig 3 in this article http://arxiv.org/pdf/1411.4389v4.pdf , the author said the label are determined by averaging the softmax of each time steps, this confuse me, can you tell me how to average the softmax of each time steps? and return sequences should to be set False or True?? Very thanks! is my following code is right?

model = Sequential()
model.add(TimeDistributed(Convolution2D(8, 5, 5, border_mode='valid'), input_shape=(maxToAdd, 1, size1, size2)))
model.add(TimeDistributed(MaxPooling2D(pool_size=(2, 2),border_mode='valid')))
model.add(Activation('relu'))
model.add(TimeDistributed(Flatten()))
model.add(LSTM(output_dim=64, return_sequences=True))
model.add(Dropout(.5))
model.add(TimeDistributed(Dense(1)))
model.add(Activation('sigmoid'))
rmsprop = RMSprop()
model.compile(loss='mean_squared_error', optimizer=rmsprop, metrics=['accuracy'])

[marouanebr]

Hi, I don't know if you are still interested, but because I had the same problem, I thought maybe I would share my implementation: model.add(LSTM(output_dim=64, return_sequences=True)) model.add(TimeDistributed(Dense(1, activation='softmax')) model.add(GlobalAveragePooling1D()) The return_sequences must be set True, so you could average the softmax output on the time axis. I hope this helps ;)

[shamanez]

@marouanebr Can you share the full code ?

[marouanebr]

I'm sorry it has been a long time since I implemented it, so I don't have the code anymore. But I believe if you replace the last three lines of the original code with mine, it should work.