Coding - Architectures - AlexNet

[NicolaBernini]

Overview

Coding AlexNet in many ways and using different frameworks

[NicolaBernini]

AlexNet in Keras

Building AlexNet from a simplified vectorial representation

This allows to build AlexNet specifying only 2 vectors of integers

• the vector defining the backend, the only free param exposed is the number of output filters
• the vector defining the frontend, the only free param exposed is the number of units

All the other params so

• for the linear computation part, the kernel sizes, strides, padding, …
• for the non linear computation part, the activation type
• for the regularization type, the regularization approach and its params

are internal details which do not get exposed

Defining the class

class AlexNet: 
  def __init__(self, input_shape=(224,224,3), dropout_perc=0.4): 
    self.input_shape = input_shape 
    self.dropout_perc = dropout_perc

  def start(self): 
    m = Sequential()
    m.add(InputLayer(input_shape=self.input_shape))
    return m

  def front_conv_layer(self,m, num_filters): 
    m.add(Conv2D(filters=num_filters, kernel_size=(11,11), strides=(4,4), padding='valid'))
    m.add(Activation('relu'))
    m.add(MaxPool2D(pool_size=(2,2), strides=(2,2), padding='valid'))

  def inner_conv_layer(self,m, num_filters): 
    m.add(Conv2D(filters=num_filters, kernel_size=(3,3), strides=(1,1), padding='valid'))
    m.add(Activation('relu'))

  def inner_dense(self,m, num_units): 
    m.add(Dense(num_units))
    m.add(Activation('relu'))
    m.add(Dropout(self.dropout_perc))

  def last_dense(self,m, num_units): 
    m.add(Dense(num_units))
    m.add(Activation('softmax'))

  # Builds an AlexNet from a simple vector representation defining the number of filters for Conv Backend and the number of Units for Dense Frontend 
  def build_from_vector(self,be=[96,256,384,384,256], fe=[4096,1000,17]): 
    m = self.start() 
    self.front_conv_layer(m, be[0])
    for i in range(1, len(be)): 
      if i<(len(be)-1): 
        self.inner_conv_layer(m, be[i])
      else: 
        self.front_conv_layer(m, be[0])
    
    for i in range(len(fe)): 
      if i<(len(fe)-1): 
        self.inner_dense(m, fe[i])
      else: 
        self.last_dense(m, fe[i])
    return m 

Test

a = AlexNet() 
m = a.build_from_vector() 
m.summary()

Results

Model: "sequential_5"
_________________________________________________________________
Layer (type)                 Output Shape              Param #   
=================================================================
conv2d_11 (Conv2D)           (None, 54, 54, 96)        34944     
_________________________________________________________________
activation_14 (Activation)   (None, 54, 54, 96)        0         
_________________________________________________________________
max_pooling2d_6 (MaxPooling2 (None, 27, 27, 96)        0         
_________________________________________________________________
conv2d_12 (Conv2D)           (None, 25, 25, 256)       221440    
_________________________________________________________________
activation_15 (Activation)   (None, 25, 25, 256)       0         
_________________________________________________________________
conv2d_13 (Conv2D)           (None, 23, 23, 384)       885120    
_________________________________________________________________
activation_16 (Activation)   (None, 23, 23, 384)       0         
_________________________________________________________________
conv2d_14 (Conv2D)           (None, 21, 21, 384)       1327488   
_________________________________________________________________
activation_17 (Activation)   (None, 21, 21, 384)       0         
_________________________________________________________________
conv2d_15 (Conv2D)           (None, 3, 3, 96)          4460640   
_________________________________________________________________
activation_18 (Activation)   (None, 3, 3, 96)          0         
_________________________________________________________________
max_pooling2d_7 (MaxPooling2 (None, 1, 1, 96)          0         
_________________________________________________________________
dense_4 (Dense)              (None, 1, 1, 4096)        397312    
_________________________________________________________________
activation_19 (Activation)   (None, 1, 1, 4096)        0         
_________________________________________________________________
dropout_3 (Dropout)          (None, 1, 1, 4096)        0         
_________________________________________________________________
dense_5 (Dense)              (None, 1, 1, 1000)        4097000   
_________________________________________________________________
activation_20 (Activation)   (None, 1, 1, 1000)        0         
_________________________________________________________________
dropout_4 (Dropout)          (None, 1, 1, 1000)        0         
_________________________________________________________________
dense_6 (Dense)              (None, 1, 1, 17)          17017     
_________________________________________________________________
activation_21 (Activation)   (None, 1, 1, 17)          0         
=================================================================
Total params: 11,440,961
Trainable params: 11,440,961
Non-trainable params: 0