Worse performance with CRF layer

[RH1994R]

trafficstars

At first I really want to thank you for providing a simple CRF alternative!

My problem with your CRF layer is a worse performace than without it. I really don't get why. Below there is my model:

`model = Sequential() model.add(Embedding(vocab_size, 100, weights=[embedding_vectors], input_length=max_seq_length, trainable= False)) model.add(Conv1D(1000, 1, activation=LeakyReLU(alpha=0.1))) model.add(Conv1D(200, 1, activation=LeakyReLU(alpha=0.1))) model.add(Dropout(0.2)) model.add(Bidirectional(LSTM(units=100, recurrent_dropout=0.4, dropout=0.4, return_sequences=True, kernel_regularizer=l2(0.000001)))) model.add(Dropout(0.4)) model.add(TimeDistributed(Dense(512, activation=LeakyReLU(alpha=0.1)))) model.add(Dense(n_tags, activation='softmax'))

opt = RMSprop(learning_rate=0.0008) model.compile(loss="categorical_crossentropy", optimizer=opt, metrics=["categorical_accuracy"])`

If I add your CRF layer to it, it looks like this: `input_layer = Input(shape=(max_seq_length,)) output = Embedding(vocab_size, 100, weights=[embedding_vectors], input_length=max_seq_length, trainable= False)(input_layer) sequence_mask = Lambda(lambda x: greater(x, 0))(input_layer)

output = Conv1D(1000, 1,activation=LeakyReLU(alpha=0.1))(output) output = Conv1D(200, 1,activation=LeakyReLU(alpha=0.1))(output) output = Dropout(0.2)(output) output = Bidirectional(LSTM(units=100, recurrent_dropout=0.4, dropout=0.4, return_sequences=True))(output) output = Dropout(0.4)(output) output = Dense(512, activation=LeakyReLU(alpha=0.1))(output) output = Dense(n_tags, activation=None)(output) crf = CRF(n_tags) output = crf(output, mask=sequence_mask)

model = Model(input_layer, output)

model.compile(optimizer="rmsprop", metrics=[crf.accuracy], loss=crf.neg_log_likelihood)`

Unfortunately the performance is worse. Before it was like 62% F1-Score, now it is 59%. Do you any idea why it is worse now? Did I miss anything which is has to be added for your CRF layer?

[luozhouyang]

I'm not sure what caused this worse performance. But if you can get better performance without a CRF, you just remove this CRF layer. In any case, the CRF is not necessary.