0.0% correct at expression level when validate model

[vndee]

I've trained an model with your data you pushed in Floyhub. During training progress I've seen train accuracy is around ~53% and validation accuracy around ~44%. But when I validated this model in 2016 dataset, I got 0.0% accuracy for expression level.

`

Dataset 2016

Beam width: 4

Highest Probability

Model	Token error rate	Token error rate (no special tokens)	Symbol error rate
ANHTUYEN0100	65.93%	85.32%	90.71%

Model	Correct expressions	Correct expressions (no special tokens)	Correct expressions (Symbols)
ANHTUYEN0100	0.00%	0.00%	0.00%

Best

Model	Token error rate	Token error rate (no special tokens)	Symbol error rate
ANHTUYEN0100	64.55%	82.76%	89.79%

Model	Correct expressions	Correct expressions (no special tokens)	Correct expressions (Symbols)
ANHTUYEN0100	0.00%	0.00%	0.00%

Mean

Model	Token error rate	Token error rate (no special tokens)	Symbol error rate
ANHTUYEN0100	65.70%	84.60%	90.64%

Model	Correct expressions	Correct expressions (no special tokens)	Correct expressions (Symbols)
ANHTUYEN0100	0.00%	0.00%	0.00%
`

[jungomi]

That is just not trained long enough. The accuracy on the training set should reach at least 95%, and if I remember correctly, the validation accuracy would be somewhere over 80% at this point.

Also the 44% accuracy on the validation set is referring to correctly predicted tokens, that doesn't necessarily mean that it was able to predict a single full expression correctly.

But even then, the expression recognition rate is generally low with this version of the dataset, I explained the primary reasons in https://github.com/jungomi/math-formula-recognition/issues/1#issuecomment-474773491.

[vndee]

Can you tell me how many epoch did you train? When I trained accuracy seem didn't increase after 100 epochs.

[jungomi]

You probably need to adjust the learning rate, including the options of the learning rate decay, as by default it reduces the learning rate by a factor of 10 every 20 epochs.

I have done some experiments with different, slightly modified architecture (e.g. LSTM instead of a GRU), and in general I had used all default parameters except for the learning rate, which was mostly at 1e-2 at the beginning for about 100 - 150 epochs and then it has been lowered by a factor of 10 every 50 epochs, and I've stopped at about 250 epochs.

The --lr-epochs option defines after how many epochs the learning rate gets reduced by the factor of the --lr-factor (it is multiplicative, i.e. lr = lr * lr_factor)