Support for multi-input and merge layer?

[blackarrow3542]

Hi, I find this project very useful and interesting! Thanks a lot! Will you add support for multi-input and merge layer?

Thanks

[pplonski]

Hi, yes I want to support multi-input and merge layer. Do you have an example of Keras network that you want to use, a reference would be great.

[blackarrow3542]

Hi, thanks for the fast reply!

For example some model with merge layer like resnet and googlenet? https://github.com/fchollet/keras/blob/master/keras/applications/resnet50.py https://github.com/fchollet/keras/blob/master/keras/applications/inception_v3.py

[spurihwr]

Hi, Thanks for the project. I find it quite useful as I am trying to deploy keras model in cpp. I came across a small bug in your keras_model.cc file in function: std::vector< std::vector > keras::conv_single_depth_valid( std::vector< std::vector > const & im, std::vector< std::vector > const & k) {}

In the nested loop, a 2 is missing in front of st_x and st_y . It should be something like: for(unsigned int i = st_x; i < im.size()-2st_x; ++i) { for(unsigned int j = st_y; j < im[0].size()-2st_y; ++j) { ....

Thank you so much. I was able to get the correct prediction by changing this.

[pplonski]

Hi @spurihwr ! In conv_single_depth_valid the st_x and st_y values are borders, so you need to add one border at start and one at end, something like thisfor(unsigned int i = st_x; i < im.size()-st_x; ++i) - this is strange, that if you changed this you started to get correct predictions. I'm going to add better testing for dumping, so I'll look closer on in. Could you share your network for testing? (is it large?)

[spurihwr]

Hi @pplonski !! Thanks for your reply. Actually, with your original code, there was a segmentation fault and that is because the loop was accessing outside the size of image in y[i - st_x][j - st_y] += k[k.size() - k1 - 1][k[0].size() - k2 - 1] * im[i - st_x + k1][j - st_y + k2]; With im.size() = 8 and kernal size k1=k2=4, im[i-st_x+k1] goes out of range at i=6. My network is something like this: model = Sequential() model.add(Convolution2D(4, 3, 3, border_mode='valid', input_shape=(1,8,8))) model.add(Activation('relu')) model.add(Convolution2D(4, 3, 3, border_mode='valid')) model.add(Activation('relu')) model.add(MaxPooling2D(pool_size=(2,2))) model.add(Dropout(0.25)) model.add(Flatten()) model.add(Dense(64)) model.add(Activation('relu')) model.add(Dropout(0.5)) model.add(Dense(num_classes)) model.add(Activation('softmax'))

[pplonski]

for kernel size = 4, indeed there can be a problem but kernels are odd (usually, I haven't seen even kernel size ??). So for kernel size = 3, st_ will be 1, and for i=6, im[i-st_x+k1] will be working, and for kernel size = 5, st_x will be 2, and i=6 will be not considered (because border mode is valid). Does it make sense?

[spurihwr]

Yes. I was initially using a kernal size of 4 which created this problem. As I am new to CNN, I was not aware of this that kernal should be odd. Thank you so much. The code is running fine. I want to use Conv1D for some task. So i will try to implement that as I see that it is not in the code and will update you here abt that. Thanks :)

[xdtl]

Do you have any suggestions for supporting multi-input and merge layer? Thanks!

[blackarrow3542]

@xdtl Hi, since keras2cpp doesn't support some neural network models, an alternative solution is to build Tensorflow with makefile to implement a Tensorflow model in an existing c++ project. https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/makefile

[xdtl]

Thanks for your reply! Does it support Windows?

[blackarrow3542]

@xdtl

Maybe you can try this? https://github.com/tensorflow/tensorflow/blob/master/tensorflow/contrib/cmake And then use TensorFlow with its c++ API https://www.tensorflow.org/api_docs/cc/.

[xdtl]

Thanks! I will look into this... It would still be great if there is a light weighted framework that can deploy trained Keras models for prediction.