Machine Learning Courses

An attempt to categorize all open-source machine learning courses.

General

Introduction to Deep Learning

• Website: https://sebastianraschka.com/blog/2021/dl-course.html

NYU Deep Learning

• Description: This course concerns the latest techniques in deep learning and representation learning, focusing on supervised and unsupervised deep learning, embedding methods, metric learning, convolutional and recurrent nets, with applications to computer vision, natural language understanding, and speech recognition.
• Website: https://atcold.github.io/pytorch-Deep-Learning/
• Videos: https://www.youtube.com/playlist?list=PLLHTzKZzVU9e6xUfG10TkTWApKSZCzuBI
• Videos (2020): https://www.youtube.com/watch?v=0bMe_vCZo30&list=PLLHTzKZzVU9eaEyErdV26ikyolxOsz6mq

MIT 6.S191 Introduction to Deep Learning

• Description: MIT's introductory course on deep learning methods with applications to computer vision, natural language processing, biology, and more! Students will gain foundational knowledge of deep learning algorithms and get practical experience in building neural networks in TensorFlow. Course concludes with a project proposal competition with feedback from staff and panel of industry sponsors. Prerequisites assume calculus (i.e. taking derivatives) and linear algebra (i.e. matrix multiplication), we'll try to explain everything else along the way! Experience in Python is helpful but not necessary.
• Website: http://introtodeeplearning.com/

DS-GA 1008 Deep Learning

• Description: This course concerns the latest techniques in deep learning and representation learning, focusing on supervised and unsupervised deep learning, embedding methods, metric learning, convolutional and recurrent nets, with applications to computer vision, natural language understanding, and speech recognition.
• Website: https://cds.nyu.edu/deep-learning/

Practical Deep Learning for Coders

• Description: After finishing this course you will know how to train models that achieve state-of-the-art results in: computer vision, including image classification (e.g., classifying pet photos by breed), and image localization and detection (e.g., finding where the animals in an image are); natural language processing (NLP), including document classification (e.g., movie review sentiment analysis) and language modeling; tabular data (e.g., sales prediction) with categorical data, continuous data, and mixed data, including time series; collaborative filtering (e.g., movie recommendation). You will also learn: how to turn your models into web applications, and deploy them; why and how deep learning models work, and how to use that knowledge to improve the accuracy, speed, and reliability of your models; the latest deep learning techniques that really matter in practice, how to implement stochastic gradient descent and a complete training loop from scratch; and, how to think about the ethical implications of your work, to help ensure that you're making the world a better place and that your work isn't misused for harm. Here are some of the techniques covered: random forests and gradient boosting; affine functions and nonlinearities; parameters and activations; random initialization and transfer learning; SGD, Momentum, Adam, and other optimizers; convolutions; batch normalization; dropout; data augmentation; weight decay; image classification and regression; entity and word embeddings; recurrent neural networks (RNNs); segmentation; and much more!
• Lecture Videos: https://course.fast.ai/
• Companion Book: https://github.com/fastai/fastbook
• Lecture Videos (2019): https://course19.fast.ai/

CS182: Designing, Visualizing and Understanding Deep Neural Networks

• Description: Broad overview of deep learning topics, including: neural network architectures, optimization algorithms, applications in vision and NLP, reinforcement learning, and advanced topics.
• Website: https://cs182sp21.github.io/
• Lecture Videos: https://www.youtube.com/playlist?list=PL_iWQOsE6TfVmKkQHucjPAoRtIJYt8a5A

CS229 Machine Learning

• Description: CS229 provides a broad introduction to statistical machine learning (at an intermediate / advanced level) and covers supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance tradeoffs, practical ); and reinforcement learning among other topics. The structure of the summer offering enables coverage of additional topics, places stronger emphasis on the mathematical and visual intuitions, and goes deeper into the details of various topics.
• Website: http://cs229.stanford.edu/syllabus-summer2019.html
• Lecture Videos (2019): https://www.youtube.com/playlist?list=PLoROMvodv4rNH7qL6-efu_q2_bPuy0adh
• Cheatsheets: https://stanford.edu/~shervine/teaching/
• Lecture Videos (2018): https://www.youtube.com/playlist?list=PLoROMvodv4rMiGQp3WXShtMGgzqpfVfbU

Deep Learning (DeepMind and UCL)

• Lecture Videos: https://www.youtube.com/watch?v=7R52wiUgxZI&list=PLqYmG7hTraZCDxZ44o4p3N5Anz3lLRVZF
• Lecture Videos (2018): https://www.youtube.com/watch?v=iOh7QUZGyiU&list=PLqYmG7hTraZCkftCvihsG2eCTH2OyGScc

Computer Vision

Natural Language Processing

CS224N: Natural Language Processing with Deep Learning

• Description: Natural language processing (NLP) is a crucial part of artificial intelligence (AI), modeling how people share information. In recent years, deep learning approaches have obtained very high performance on many NLP tasks. In this course, students gain a thorough introduction to cutting-edge neural networks for NLP.
• Website: https://web.stanford.edu/class/cs224n/
• Lecture Videos: https://www.youtube.com/playlist?list=PLoROMvodv4rOhcuXMZkNm7j3fVwBBY42z

NLP Course | For You

• Description: NLP course covering: word embeddings, text classification, language modeling, sequence-to-sequence models, attention, and transfer learning.
• Website: https://lena-voita.github.io/nlp_course.html

Unsupervised Learning

CS294-158-SP20: Deep Unsupervised Learning

• Description: This course will cover two areas of deep learning in which labeled data is not required: Deep Generative Models and Self-supervised Learning. Recent advances in generative models have made it possible to realistically model high-dimensional raw data such as natural images, audio waveforms and text corpora. Strides in self-supervised learning have started to close the gap between supervised representation learning and unsupervised representation learning in terms of fine-tuning to unseen tasks. This course will cover the theoretical foundations of these topics as well as their newly enabled applications.
• Website: https://sites.google.com/view/berkeley-cs294-158-sp20/home
• Lecture Videos: https://www.youtube.com/playlist?list=PLwRJQ4m4UJjPiJP3691u-qWwPGVKzSlNP
• Website (2019): https://sites.google.com/view/berkeley-cs294-158-sp19/home

Reinforcement Learning

Introduction to Reinforcement Learning (UCL)

• Website: https://www.davidsilver.uk/teaching/
• Lecture Videos: https://www.youtube.com/playlist?list=PLqYmG7hTraZDM-OYHWgPebj2MfCFzFObQ

Reinforcement Learning (DeepMind and UCL)

• Lecture Videos: https://www.youtube.com/watch?v=TCCjZe0y4Qc&list=PLqYmG7hTraZDVH599EItlEWsUOsJbAodm
• Lecture Videos (2020): https://www.youtube.com/watch?v=7R52wiUgxZI&list=PLqYmG7hTraZCDxZ44o4p3N5Anz3lLRVZF
• Lecture Videos (2018): https://www.youtube.com/watch?v=ISk80iLhdfU&list=PLqYmG7hTraZBKeNJ-JE_eyJHZ7XgBoAyb

Mathematics and Theory

Mathematics for Machine Learning

• Description: For a lot of higher level courses in Machine Learning and Data Science, you find you need to freshen up on the basics in mathematics - stuff you may have studied before in school or university, but which was taught in another context, or not very intuitively, such that you struggle to relate it to how it’s used in Computer Science. This specialization aims to bridge that gap, getting you up to speed in the underlying mathematics, building an intuitive understanding, and relating it to Machine Learning and Data Science.
• Website: https://www.coursera.org/specializations/mathematics-machine-learning

Deep learning theory lecture notes

• Description: Aim to provide simplified proofs over what appears in the literature, ideally reducing difficult things to something that fits in a single lecture. Primarily focused on a classical perspective of achieving a low test error for binary classification with IID data via standard (typically ReLU) feedforward networks.
• Website: https://mjt.cs.illinois.edu/dlt/

UC Berkeley CS W182 / 282A: Designing, Visualizing and Understanding Deep Neural Networks

• Website: https://cs182sp21.github.io/
• Lecture Videos: https://www.youtube.com/playlist?list=PL_iWQOsE6TfVmKkQHucjPAoRtIJYt8a5A

Applications

CS5785: Applied Machine Learning

• Description: Learn and apply key concepts of modeling, analysis and validation from Machine Learning, Data Mining and Signal Processing to analyze and extract meaning from data. Implement algorithms and perform experiments on images, text, audio and mobile sensor measurements. Gain working knowledge of supervised and unsupervised techniques including classification, regression, clustering, feature selection, association rule mining and dimensionality reduction.
• Website: https://github.com/kuleshov/cornell-cs5785-applied-ml
• Lecture Videos: https://www.youtube.com/playlist?list=PL2UML_KCiC0UlY7iCQDSiGDMovaupqc83

Full Stack Deep Learning

• Description: There are many great courses to learn how to train deep neural networks. However, training the model is just one part of shipping a deep learning project. This course teaches full-stack production deep learning: formulating the problem and estimating project cost; finding, cleaning, labeling, and augmenting data; picking the right framework and compute infrastructure; troubleshooting training and ensuring reproducibility; deploying the model at scale.
• Website: https://fullstackdeeplearning.com/

Miscellaneous

CS839 Special Topics in Deep Learning

• Description: In recent years, deep learning has enabled huge progress in many domains including computer vision, speech, NLP, and robotics. This class is designed to help students develop a deeper understanding of deep learning and explore new research directions and applications of AI/deep learning. The course goes in depth on cutting-edge topics within deep learning and their applications, including recent advances in neural architecture design, robustness and reliability of neural networks under adversarial and anomalous attack, learning with less supervision, deep generative modeling, theoretical understanding of deep learning, as well as explaining black-box deep learning models to enhance their transparency. It assumes that students already have a basic understanding of deep learning.
• Website: http://pages.cs.wisc.edu/~sharonli/courses/cs839_fall2020/index.html
• Lecture Videos: https://www.youtube.com/playlist?list=PLKvO2FVLnI9SYLe1umkXsOfIWmEez04Ii

CSC2541 Topics in Machine Learning: Neural Net Training Dynamics

• Description: Neural nets have achieved amazing results over the past decade in domains as broad as vision, speech, language understanding, medicine, robotics, and game playing. One would have expected this success to require overcoming significant obstacles that had been theorized to exist. After all, the optimization landscape is nonconvex, highly nonlinear, and high-dimensional, so why are we able to train these networks? In many cases, they have far more than enough parameters to memorize the data, so why do they generalize well? This class is about developing the conceptual tools to understand what happens when a neural net trains.
• Website: https://www.cs.toronto.edu/~rgrosse/courses/csc2541_2021/

Other Resources

Books

• https://explained.ai/matrix-calculus/
• deeplearningbook.org
• d2l.ai
• https://mml-book.github.io/