When training classifier_train.py with my own dataset, the loss never drops and the acc is very low!

[Supzekun]

I trained the classifier on my own dataset, which has a total of 100 images divided into 5 classes with 20 images in each class. Each image is in the form of label1_xxx.jpg, label2_xxx.jpg named. But when training, the loss is always big and not decreasing, the acc is always low and my batch_size is 4. Why is this? Is the dataset too small?