Machine Learning Course (Fall 2018)

Codes and Projects for Machine Learning Course, University of Tabriz.

• Course Website
• YouTube Videos (in Persian)

Contents:

Chapter 1: Introduction (video)

• download slides in Persian (pdf)

Supervised Learning

Chapter 2: Regression

• Linear regression
• Gradient descent algorithm (video)
• Multi-variable linear regression
• Polynomial regression (video)
• Normal equation
• Locally weighted regression
• Probabilistic interpretation (video)
• Download slides in Persian (pdf)

Chapter 3: Python and NumPy

• Python basics
• Creating vectors and matrices in numpy
• Reading and writing data from/to files
• Matrix operations (video)
• Colon (:) operator
• Plotting using matplotlib (video)
• Control structures in python
• Implementing linear regression cost function (video)

Chapter 4: Logistic Regression (video)

• Classification and logistic regression
• Probabilistic interpretation
• Logistic regression cost function
• Logistic regression and gradient descent
• Multi-class logistic regression
• Advanced optimization methods
• Download slides in Persian (pdf)

Furthur Reading

• Artificial Intelligence: A Modern Approach (3rd Edition), pages 725-727
• An Introduction to Statistical Learning: with Applications in R, pages 130-137
• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, pages 119-128

Chapter 5: Regularization (video)

• Overfitting and Regularization
• L2-Regularization (Ridge)
• L1-Regularization (Lasso)
• Regression with regularization
• Classification with regularization
• Download slides in Persian (pdf)

Furthur Reading

• The Elements of Statistical Learning: Data Mining, Inference, and Prediction, pages 61-68

Chapter 6: Neural Networks (video)

• Milti-class logistic regression
• Softmax classifier
• Training softmax classifier
• Geometric interpretation
• Non-linear classification
• Neural Networks (video: part 2)
• Training neural networks: Backpropagation
• Training neural networks: advanced optimization methods (video: part 3)
• Gradient checking
• Mini-batch gradient descent
• Download slides in Persian (pdf)

Demo:

• Train a neural network in your browser

Related Videos:

• Step by step Implementation of a multi-layer neural network in Python
• Backpropagation algorithm: Step by step example
• Activation functions (Sigmoid, Tanh, ReLU, PReLU, Maxout) and weight initialization methods
• How to solve problems using neural networks
• Dropout: A Simple Way to Prevent Neural Networks from Overfitting
• Neurons and how they work

Free Online Books:

• Neural Networks and Deep Learning; Michael Nielsen: This book is a very good place to start learning about neural networks and deep learning.
• Deep Learning by Ian Goodfellow, Yoshua Bengio, and Aaron Courville: For a more technical review of neural networks and deep learning, I recommend this book.

Chapter 7: Support Vector Machines

• Motivation: optimal decision boundary (video: part 1)
• Support vectors and margin
• Objective function formulation: primal and dual
• Non-linear classification: soft margin (video: part 2)
• Non-linear classification: kernel trick
• Multi-class SVM
• Download slides in persian (pdf)

Demo:

• SVM in javascript

Furthur Reading

Unsupervided Learning

Chapter 8: Clustering (video)

• Supervised vs unsupervised learning
• Clustering
• K-Means clustering algorithm (demo)
• Determining number of clusters: Elbow method
• Postprocessing methods: Merge and Split clusters
• Bisectioning clustering
• Hierarchical clustering
• Application 1: Clustering digits
• Application 2: Image Compression
• Download slides in Persian (pdf)

Chapter 9: Dimensionality Reduction and PCA (video)

• Introduction to PCA
• PCA implementation in python
• PCA Applications
• Singular Value Decomposition (SVD)
• Downloas slides in Persian (pdf)

Chapter 10: Anomally Detection (video: Part 1, Part 2)

• Intoduction to anomaly detection
• Some applications (security, manufacturing, fraud detection)
• Anoamly detection using probabilitic modelling
• Uni-variate normal distribution for anomaly detection
• Multi-variate normal distribution for anomaly detection
• Evaluation measures (TP, FP, TN, FN, Precision, Recall, F-score)
• Anomaly detection as one-class classification
• Classification vs anomaly detection
• Download slides in Persian (pdf)

Chapter 11: Recommender Systems (video)

• Introduction to recommender systems
• Collaborative filtering approach
• User-based collaborative filtering
• Item-based collaborative filtering
• Similarity measures (Pearson, Cosine, Euclidian)
• Cold start problem
• Singular value decomposition
• Content-based recommendation
• Cost function and minimization
• Download slides in Persian (pdf)

Other Useful Resources

• Optimization: Convex Optimization, Stephan Boyd, Stanford
• Linear algebra: pdf
• Calculus: Khan Accademy
• Probability: Khan Accademy

Assignments:

1. Regression and Gradient Descent
2. Classification, Logistic Regression and Regularization
3. Multi-Class Logistic Regression
4. Neural Networks Training
5. Neural Networks Implementing
6. Clustering
7. Dimensionallity Reduction and PCA
8. Recommender Systems