predicted classes - custom dataset

[Aishaj]

Hi,

I retrained the model with my own dataset. I created one class for the wanted word (only one wanted word in my case) and another class for unwanted words which has random files from the original dataset. I also changed the duration of the file to 1500 milliseconds. The problems are: 1- The predicted class is always one class. 2- The model accuracy is different each time I run the model. One is around 83% and other time it is around 17%.

The data is balanced but couldn't figure out what is the problem. `24/24 [==============================] - 3s 80ms/step - loss: 0.8777 - accuracy: 0.6862 - val_loss: 1.0598 - val_accuracy: 0.6667

Epoch 2/12 24/24 [==============================] - 2s 65ms/step - loss: 0.4955 - accuracy: 0.8997 - val_loss: 1.0245 - val_accuracy: 0.6667

Epoch 3/12 24/24 [==============================] - 2s 65ms/step - loss: 0.3274 - accuracy: 0.9115 - val_loss: 0.9986 - val_accuracy: 0.6667

Epoch 4/12 24/24 [==============================] - 2s 68ms/step - loss: 0.2524 - accuracy: 0.9258 - val_loss: 0.9799 - val_accuracy: 0.6667

Epoch 5/12 24/24 [==============================] - 2s 66ms/step - loss: 0.2001 - accuracy: 0.9583 - val_loss: 0.9677 - val_accuracy: 0.6667

Epoch 6/12 24/24 [==============================] - 4s 154ms/step - loss: 0.1572 - accuracy: 0.9831 - val_loss: 0.9584 - val_accuracy: 0.6667

Epoch 7/12 24/24 [==============================] - 2s 67ms/step - loss: 0.1263 - accuracy: 0.9831 - val_loss: 0.9561 - val_accuracy: 0.6667

Epoch 8/12 24/24 [==============================] - 2s 68ms/step - loss: 0.1007 - accuracy: 0.9909 - val_loss: 0.9609 - val_accuracy: 0.6667

Epoch 9/12 24/24 [==============================] - 2s 68ms/step - loss: 0.0707 - accuracy: 0.9987 - val_loss: 0.9670 - val_accuracy: 0.6667

Epoch 10/12 24/24 [==============================] - 2s 68ms/step - loss: 0.0557 - accuracy: 0.9987 - val_loss: 0.9830 - val_accuracy: 0.6667

Epoch 11/12 24/24 [==============================] - 2s 69ms/step - loss: 0.0523 - accuracy: 0.9948 - val_loss: 1.0069 - val_accuracy: 0.8333

Epoch 12/12 24/24 [==============================] - 2s 67ms/step - loss: 0.0405 - accuracy: 0.9974 - val_loss: 1.0212 - val_accuracy: 0.1667

1/1 [==============================] - 0s 62ms/step - loss: 1.0094 - accuracy: 0.1667 Final test accuracy: 16.67%

Running testing on validation set... Preidcted tf.Tensor([0 0 0 0 0 0], shape=(6,), dtype=int64) [1 2 0 2 2 2] Validation accuracy = 16.67%(N=6) Running testing on test set... predicted tf.Tensor([0 0 0 0 0 0], shape=(6,), dtype=int64) [2 2 0 1 2 2] [[1 0 0] [1 0 0] [4 0 0]] Test accuracy = 16.67%(N=6)`

[Aishaj]

Hi Richard @Burton2000 Is there anything you would recommend trying?

[Burton2000]

Just some initial things:

You mentioned you are using 1500ms length clips which is different from the default 1000, have you set the clip_duration_ms parameter correctly when training? Also, are all your clips at a sampling rate of 16000 which is what we normally expect. You would have to change the the sample_rate parameter when training if using a new sample rate.

How many examples in your different categories that you added, are the datasets balanced etc. this would have an affect on the training if you have one class with loads more samples than another.

[Aishaj]

Thank you for your reply @Burton2000 . Yes, I already did all that you mentioned. and yes the data is balanced. Not sure what could be the problem