Val accuracy did not improved from 0.31467 for custom dataset

[AnushaMehta]

Hello, I am using custom dataset with 3000 images(Having 3 classes, each contain 1000 images). The dataset is divided to 2250 training and 750 testing examples. The input to the model is (32,32,3). I trained the model using learning rate 0.01, 0.001, 0.0001 respectively with epoch 50, batchsize 128 and adam optimizer. But the val_capsnet_accuracy did not improved from 0.31467. Why it is happened? Should I increase number of images in my dataset? Or any other changes in parameters or network can help me?

[innat]

Hardly anyone can help with the provided information. However, a few things you should do such as

• grayscale your images.
• Use learning rate let's say 0.001 (don't panic, do experiment later).
• add a couple of convolution layer at the beginning.
• ensure the proper preprocessing steps (such as correct labeling, proper distribution of each dataset, normalize (boost your computation time) ...etc.
• the number of the capsule and their dimensions.

Here, CapNet is a single input and multi-output network. So, you can eliminate the decoder part and observe the result.

[reneplonus]

Hello Xifeng Guo,

we encountered a similar problem as AnushaMehta before and wondered if you might be able to help us out. We have 28k training images and 1800 test images which are divided into 35 classes (we are working with plancton images). We already tried to add layers from different pre-trained models as convolutional base (VGG16; InceptionResNetV2;...), but couldn't manage to achive more than 20% test accuracy (30% validation accuracy). With a finetuned CNN model (with Xception as convolutional base) we achived 82% test accuracy and expected the CapsNet to be superior. Is there anything wrong with our implementation of the CapsNet? The entire code is below. We did not change anything in functions and classes we omitted (like Mask or PrimaryCap). We would really appreciate your help in this matter!

Thanks for any help in advance.

Based on code by Kevin Mader (https://www.kaggle.com/kmader/capsulenet-on-mnist/notebook)

import keras.backend as K import tensorflow as tf from keras import initializers, layers from keras.layers import Conv2DTranspose from keras import models from keras.applications import VGG16 import keras

conv_base = VGG16(include_top=False, weights='imagenet', input_tensor=None, input_shape=(240, 240, 3), pooling=None,classes=35)

conv_base.trainable = True

set_trainable = False for layer in conv_base.layers: if layer.name == 'block2_conv2': set_trainable = True if set_trainable: layer.trainable = True else: layer.trainable = False

mylayers = conv_base.layers[:11]

def CapsNet(input_shape, n_class, num_routing): """ A Capsule Network on MNIST. :param input_shape: data shape, 4d, [None, width, height, channels] :param n_class: number of classes :param num_routing: number of routing iterations :return: A Keras Model with 2 inputs and 2 outputs """ x = layers.Input(shape=input_shape)

# Layer 1: Just a conventional Conv2D layer

l1 = mylayers[0](x)
l2 = mylayers[1](l1)
bn = layers.BatchNormalization()(l2)
l3 = mylayers[2](bn)
l4 = mylayers[3](l3)
l5 = mylayers[4](l4)
l6 = mylayers[5](l5)
l7 = mylayers[6](l6)
l8 = mylayers[7](l7)
l9 = mylayers[8](l8)
m1 = layers.MaxPool2D((2, 2))(l9)
l10 = layers.Conv2D(180, (2, 2), activation = 'relu', padding = 'valid')(m1)
m2 = layers.MaxPool2D((2, 2))(l10)
do = layers.Dropout(0.5)(m2)

# Layer 2: Conv2D layer with `squash` activation, then reshape to [None, num_capsule, dim_vector]
primarycaps = PrimaryCap(do, dim_vector=24, n_channels=20, kernel_size=6, strides=2, padding='valid')

# Layer 3: Capsule layer. Routing algorithm works here.
digitcaps = CapsuleLayer(num_capsule=n_class, dim_vector=16, num_routing=num_routing, name='digitcaps')(primarycaps)
# dim_vector war 16

# Layer 4: This is an auxiliary layer to replace each capsule with its length. Just to match the true label's shape.
# If using tensorflow, this will not be necessary. :)
out_caps = Length(name='out_caps')(digitcaps)

# Decoder network.
y = layers.Input(shape=(n_class, ))
masked = Mask()([digitcaps, y])  # The true label is used to mask the output of capsule layer
x_recon = layers.Dense(51200, activation='relu')(masked)
x_reshape = layers.Reshape((20, 20, 128))(x_recon)
conv1_1 = layers.Conv2D(50, (2,2), activation = 'relu', padding = "SAME")(x_reshape)
upsampling_1 = Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(conv1_1)
conv1_a = layers.Conv2D(100, (3,3), activation = 'relu', padding = "SAME")(upsampling_1)
upsampling_a = Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(conv1_a)
conv1_2 = layers.Conv2D(150, (3,3), activation = 'relu', padding = "SAME")(upsampling_a)
upsampling_2 = Conv2DTranspose(100, 3, padding='same', activation='relu', strides=(3, 3))(conv1_2)
conv1_b = layers.Conv2D(200, (2,2), activation = 'relu', padding = "SAME")(upsampling_2)
upsampling_b = Conv2DTranspose(100, 3, padding='same', activation='relu', strides=(1, 1))(conv1_b)
decoded_outputs = Conv2DTranspose(3, 3, padding='same', activation='relu')(upsampling_b)

# two-input-two-output keras Model
return models.Model([x, y], [out_caps, decoded_outputs])

model = CapsNet(input_shape=[240, 240, 3], n_class=35, num_routing=5) model.summary()

from keras.preprocessing.image import ImageDataGenerator

import matplotlib.pyplot as plt

base_dir = '/home/bigdatauser/Jan/training_set_2_step_2/' train_dir = base_dir + 'train' validation_dir = base_dir + 'validation' test_dir = base_dir + 'test'

def brightimgs(img): img2 = (img*2)**2 img2 = img2/255 return img2 # to get more contrast / remove particles in the images

train_datagen = ImageDataGenerator(rescale = 1./255, rotation_range=40, width_shift_range=0.2, height_shift_range=0.2, shear_range=0.2, zoom_range=0.2, horizontal_flip=True, fill_mode='nearest', preprocessing_function = brightimgs)

test_datagen = ImageDataGenerator(rescale = 1./255, preprocessing_function = brightimgs)

def my2generators(directory, target_size, batch_size): generator_1 = train_datagen.flow_from_directory( directory = directory, target_size = target_size, batch_size = batch_size, class_mode = 'categorical', shuffle = True) while True: a = generator_1.next()[0] b = generator_1.next()[1] yield [a, b], [b, a]

def my2generators2(directory, target_size, batch_size): generator_1 = test_datagen.flow_from_directory( directory = directory, target_size = target_size, batch_size = batch_size, class_mode = 'categorical', shuffle = True) while True: a = generator_1.next()[0] b = generator_1.next()[1] yield [a, b], [b, a]

model.compile(optimizer='adam', loss=[margin_loss, 'mse'], loss_weights=[0.5, 0.5], # since we have few images with overlapping objects, we weighted the losses differently metrics={'out_caps': 'accuracy'})

train_generator = my2generators(train_dir, (240, 240), 1)

validation_generator = my2generators2(validation_dir, (240, 240), 1)

history = model.fit_generator(generator=train_generator, steps_per_epoch=18000, epochs=100, validation_data=validation_generator, validation_steps = 3000, callbacks = [keras.callbacks.ReduceLROnPlateau()])

loss = history.history['loss'] out_caps_loss = history.history['out_caps_loss'] val_loss = history.history['val_loss'] conv2d_loss = history.history['conv2d_transpose_5_loss'] out_caps_acc = history.history['out_caps_acc'] val_out_caps_loss = history.history['val_out_caps_loss'] val_conv2d_loss = history.history['val_conv2d_transpose_5_loss'] val_out_caps_acc = history.history['val_out_caps_acc']

plt.plot(range(len(loss)), loss, 'b') plt.plot(range(len(loss)), out_caps_loss, 'b') plt.plot(range(len(loss)), val_loss, 'b') plt.plot(range(len(loss)), conv2d_loss, 'b') plt.plot(range(len(loss)), out_caps_acc, 'b') plt.plot(range(len(loss)), val_out_caps_loss, 'b') plt.plot(range(len(loss)), val_conv2d_loss, 'b') plt.plot(range(len(loss)), val_out_caps_acc, 'b')

def my2generators_for_test(directory, target_size, batch_size): generator_1 = test_datagen.flow_from_directory( directory = directory, target_size = target_size, batch_size = batch_size, class_mode = 'categorical', shuffle = False) while True: a = generator_1.next()[0] b = generator_1.next()[1] yield [a, b], [b, a]

test_generator = my2generators_for_test(test_dir, (240, 240), 1)

val_loss, val_out_caps_loss, val_conv2d_transpose_10_loss, val_out_caps_acc = model.evaluate_generator(test_generator, steps=960)

print(val_loss, val_out_caps_loss, val_conv2d_transpose_10_loss, val_out_caps_acc)

model.save('/home/bigdatauser/Jan/0001VPR/py_scripte/Tryout_Scripts/CapsNet/CapsNet_FlowCam_classifier_step_2_conv_base_inception_240_px_outcaps.h5')

[reneplonus]

Hello AnushaMehta,

 

we tried cifar and it didn't work too well either. When trying cifar we also used 32,32 as input shape and not 240,240 (we tried different input sizes with our data and also different or no augmentation)

 

Gesendet: Donnerstag, 04. April 2019 um 15:02 Uhr Von: "AnushaMehta" [email protected] An: XifengGuo/CapsNet-Keras [email protected] Cc: reneplonus [email protected], Mention [email protected] Betreff: Re: [XifengGuo/CapsNet-Keras] Val accuracy did not improved from 0.31467 for custom dataset (#92)

@reneplonus Can we use cifar10 dataset with the same model used by you ? If yes, then what is the process of converting (32,32,3) input shape to (240,240,3)?

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread.

[AnushaMehta]

Hello AnushaMehta, we tried cifar and it didn't work too well either. When trying cifar we also used 32,32 as input shape and not 240,240 (we tried different input sizes with our data and also different or no augmentation) Gesendet: Donnerstag, 04. April 2019 um 15:02 Uhr Von: "AnushaMehta" [email protected] An: XifengGuo/CapsNet-Keras [email protected] Cc: reneplonus [email protected], Mention [email protected] Betreff: Re: [XifengGuo/CapsNet-Keras] Val accuracy did not improved from 0.31467 for custom dataset (#92) @reneplonus Can we use cifar10 dataset with the same model used by you ? If yes, then what is the process of converting (32,32,3) input shape to (240,240,3)? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub, or mute the thread. I have tried the same code for cifar10 dataset and i am getting the accuracy almost 78%. When I used the capsule network without pretrained vgg model, I am getting accuracy almost 64%, so i guess by using pretrained model, it gives better accuracy for cifar10 dataset.

[diosguo]

I found that the Length of my caps output all become '1' or '0' after trainingg several batch, and my acc does not improve any more from that, that's interesting. but confusing me hardly