Data imbalance problem for Voice Activity Detection output block

[bisnu16cse039]

As the dataset will contain rare amount of data containing only noise data after creating noisy and clean, Voice Activity Block should suffer from class imbalance problem. When I am training with new dataset, I am suffering weird changes in VAD msse loss and validation msse loss.And I am suspecting class imbalance problem for such issue. How to deal with such cases?