itayhubara
Slower Inference time than normal network
Hi,
I tried your implementation and made some fixes so that it could be used with tensorflow 1.5 (removed the functions inside the functions) and its estimators. After a couple runs I noticed that the inference time of the binary network on mnist is slower than the normal mnist network while being less accurate .
Binary network: Binarized_Conv_32,HardTanh,Binarized_Conv_64,HardTanh, Binarized_Affine 1024, HardTanh, Binarized Affine 10 Normal network: CV32, Relu, CV64, Relu, Dense 1024, Relu, Dense 10
The normal network runs in about 330 ms for 100 iterations The binary network runs in about 402 ms for 100 iterations
I am running the networks on a GTX 1060 and a intel 7700hq.
Have you ever encountered this kind of issues or should I move to Torch to try binary networks ?
The reason I didn't used your framework is because I can't get the concat function to work and I'd like to make an inception like network using binary layers.
This is the kind of model I tried with your implementation: model = Sequential([ BinarizedWeightOnlySpatialConvolution(64,3,3,1,1, padding='VALID', bias=False), BatchNormalization(), HardTanh(), BinarizedSpatialConvolution(64,3,3, padding='SAME', bias=False), BatchNormalization(), HardTanh(), BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False), BatchNormalization(), HardTanh(), Concat([ Sequential([ BinarizedSpatialConvolution(96,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh() ]), Sequential([ BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh() ]) ]), Concat([ Sequential([ BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh(), BinarizedSpatialConvolution(196,3,3, padding='SAME', bias=False), BatchNormalization(), HardTanh() ]), Sequential([ BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False), BatchNormalization(), HardTanh(), BinarizedSpatialConvolution(128,3,3, padding='SAME', bias=False), BatchNormalization(), HardTanh(), BinarizedSpatialConvolution(196,3,3, padding='SAME', bias=False), BatchNormalization(), HardTanh(), BinarizedSpatialConvolution(196,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh() ]) ]), Concat([ Sequential([ BinarizedSpatialConvolution(396,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh() ]) ]), Concat([ Sequential([ BinarizedSpatialConvolution(396,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh() ]), Sequential([ BinarizedSpatialConvolution(396,3,3, padding='SAME', bias=False), SpatialMaxPooling(2,2,2,2), BatchNormalization(), HardTanh() ]) ]), BinarizedAffine(1024, bias=False), BatchNormalization(), HardTanh(), BinarizedAffine(1024, bias=False), BatchNormalization(), HardTanh(), BinarizedAffine(10), BatchNormalization() ])
