MNIST-baselines
MNIST-baselines copied to clipboard
Published
20 hours ago •
cxy1997
cxy1997
Baseline classifiers on the polluted MNIST dataset, SJTU CS420 course project
MNIST-baselines
This repository contains various baseline models for comparison on the polluted MNIST dataset
Requirements
Usage
Create a model
- Prepare
Model_Name.jsoninconfig/ - Prepare
Model_Name.pyinmodels/ - Prepare
Trainer_Name.pyintrainers/(optional)
Train a model
python3 main.py --method Model_Name
Test a model
python3 main.py --method Model_Name --test
Trained models
Available at https://drive.google.com/open?id=1jXe2kJJLkhGH8T30juGbyUgsaDQZXISu
Polluted MNIST
Feature Extraction
A visualization of the first 15 principle components:
| Model | Accuracy / % | Model | Accuracy / % |
|---|---|---|---|
| Plain DNN | 90.72 | DNN + Dropout | 90.74 |
| DNN + Batch Normalization | 92.39 | DNN + PCA (0.99 variance) | 21.40 |
| DNN + PCA (0.95 variance) | 21.34 | DNN + PCA (0.90 variance) | 20.55 |
| DNN + ICA | 11.93 | DNN + NMF | 11.30 |
| DNN + VAE | 95.24 |
Conventional Classification Approaches
| Model | Accuracy / % | Model | Accuracy / % |
|---|---|---|---|
| Empty Model | 11.04 | Naive Bayes | 19.14 |
| Logistic Regression | 24.37 | Decision Tree (gini) | 51.24 |
| Decision Tree (entropy) | 50.91 | KNN (3 neighbours) | 78.82 |
| KNN (5 neighbours) | 78.46 | KNN (10 neighbours) | 76.40 |
| SGD | 22.30 | SVM (sigmoid kernel) | 10.84 |
| SVM (rbf kernel) | 85.94 | SVM (polynomial kernel) | 87.11 |
The training curves of the above DNN models:
Deep Convolutional Neural Networks
| Model | Accuracy / % | Model | Accuracy / % |
|---|---|---|---|
| LeNet-5 | 98.35 | MobileNetV2 | 99.63 |
| VGG-19 | 99.64 | ShuffleNetG3 | 99.65 |
| ResNet-101 | 99.71 | GoogLeNet | 99.82 |
| PreAct ResNet-152 | 99.68 | PNASNet | 99.75 |
| DenseNet-161 | 99.75 | ResNeXt-29-8x64d | 99.71 |
| DPN-92 | 99.76 | SENet-18 | 99.69 |
| MobileNet | 99.44 | CapsNet | 98.84 |
Error Analysis
cxy1997