Accuracy very low and not improving

[bhawmik]

Hi, I am using your basic LSTM architecture to recreate the chatbot. However, I am using GloVe embedding. During my training process, my Training accuracy gets stuck at very low values (0.1969) and no progress happens. I am attaching my code below. Can you tell me what can be done to improve the training?

from keras.models import Sequential from keras.layers import Embedding, Flatten, Dense, LSTM from keras.optimizers import Adam

#model.reset_states() model=Sequential() model.add(Embedding(max_words,embedding_dim,input_length=maxlen)) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.add(LSTM(units=100,return_sequences=True, kernel_initializer="glorot_normal", recurrent_initializer="glorot_normal", activation='sigmoid')) model.summary()

model.layers[0].set_weights([embedding_matrix]) model.layers[0].trainable = False

model.compile(loss='cosine_proximity', optimizer='adam', metrics=['accuracy']) #model.compile(loss='mse', optimizer='adam', metrics=['accuracy']) model.fit(x_train, y_train, epochs = 500, batch_size = 32, validation_data=(x_val,y_val))

Epoch 498/500 60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909 Epoch 499/500 60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909 Epoch 500/500 60/60 [==============================] - 0s 3ms/step - loss: -0.1303 - acc: 0.1969 - val_loss: -0.1785 - val_acc: 0.2909

Further training (on the same conversation data set ) does not improve accuracy.