Deep Learning Fundamentals: Code Materials and Exercises

This repository contains code materials & exercises for Deep Learning Fundamentals course by Sebastian Raschka and Lightning AI.

• Link to the course website: https://lightning.ai/pages/courses/deep-learning-fundamentals/
• Link to the discussion forum: https://github.com/Lightning-AI/dl-fundamentals/discussions
• Reach out to Lightning & Sebastian on social media: @LightningAI @rasbt

For other announcements, updates, and additional materials, you can follow Lightning AI and Sebastian on Twitter!

Links to the materials

Unit 1. Welcome to Machine Learning and Deep Learning [ Link to videos ]

• 1.1 What Is Machine Learning?
• 1.2 How Can We Use Machine Learning?
• 1.3 A Typical Machine Learning Workflow (The Supervised Learning Workflow)
• 1.4 The First Machine Learning Classifier
• 1.5 Setting Up Our Computing Environment
• 1.6 Implementing a Perceptron in Python
• 1.7 Evaluating Machine Learning Models
• Unit 1 exercises
  • Exercise 1: Add early-stopping to make the Perceptron more efficient
  • Exercise 2: Initialize the model parameters with small random numbers instead of 0's
  • Exercise 3: Use a learning rate for updating the weights and bias unit

Unit 2. First Steps with PyTorch: Using Tensors [ Link to videos ]

• 2.1 Introducing PyTorch
• 2.2 What Are Tensors?
• 2.3 How Do We Use Tensors in PyTorch?
• 2.4 Improving Code Efficiency with Linear Algebra
• 2.5 Debugging Code
• 2.6 Revisiting the Perceptron Algorithm
• 2.7 Seeing Predictive Models as Computation Graphs
• Unit 2 exercises
  • Exercise 1: Introducing more PyTorch functions to make your code more efficient
  • Exercise 2: Make the perceptron more efficient using matrix multiplication

Unit 3. Model Training in PyTorch [ Link to videos ]

• 3.1 Using Logistic Regression for Classification
• 3.2 The Logistic Regression Computation Graph
• 3.3 Model Training with Stochastic Gradient Descent
• 3.4 Automatic Differentiation in PyTorch
• 3.5 The PyTorch API
• 3.6 Training a Logistic Regression Model in PyTorch
• 3.7 Feature Normalization
• Unit 3 exercises
  • Exercise 1: Banknote Authentication
  • Exercise 2: Standardization

Unit 4. Training Multilayer Neural Networks [ Link to videos ]

• 4.1 Dealing with More than Two Classes: Softmax Regression
• 4.2 Multilayer Neural Networks and Why We Need Them
• 4.3 Training a Multilayer Perceptron in PyTorch
  • XOR data
  • MNIST data
• 4.4 Defining Efficient Data Loaders
• 4.5 Multilayer Neural Networks for Regression
• 4.6 Speeding Up Model Training Using GPUs
• Unit 4 exercises
  • Excercise 1: Changing the Number of Layers
  • Exercise 2: Implementing a Custom Dataset Class for Fashion MNIST

Unit 5. Organizing your PyTorch Code with Lightning [ Link to videos ]

• 5.1 Organizing Your Code with PyTorch Lightning
• 5.2 Training a Multilayer Perceptron in PyTorch Lightning
• 5.3 Computing Metrics Efficiently with TorchMetrics
• 5.4 Making Code Reproducible
• 5.5 Organizing Your Data Loaders with Data Modules
• 5.6 The Benefits of Logging Your Model Training
• 5.7 Evaluating and Using Models on New Data
• 5.8 Add functionality with callbacks
• Unit 5 exercises

Unit 6. Essential Deep Learning Tips & Tricks [ Link to videos ]

• 6.1 Model Checkpointing and Early Stopping
• 6.2 Learning Rates and Learning Rate Schedulers
• 6.3 Using More Advanced Optimization Algorithms
• 6.4 Choosing Activation Functions
• 6.5 Automating The Hyperparameter Tuning Process
• 6.6 Improving Convergence with Batch Normalization
• 6.7 Reducing Overfitting With Dropout
• 6.8 Debugging Deep Neural Networks
• Unit 6 exercises

Unit 7. Getting Started with Computer Vision [ Link to videos ]

• 7.1 Working With Images
• 7.2 How Convolutional Neural Networks Work
• 7.3 Convolutional Neural Network Architectures
• 7.4 Training Convolutional Neural Networks
• 7.5 Improving Predictions with Data Augmentation
• 7.6 Leveraging Pre-trained Models with Transfer Learning
• 7.7 Using Unlabeled Data with Self-Supervised
• Unit 7 exercises

Unit 8. Introduction to Natural Language Processing and Large Language Models [ Link to videos ]

• 8.1 Working with Text Data
• 8.2 Training Text Classifier Baseline
• 8.3. Introduction to Recurrent Neural Networks
• 8.4 From RNNas to the Transformer Architecture
• 8.5 Understanding Self-Attention
• 8.6 Large Language Models
• 8.7 Using Large Language Model for Classification
• Unit 8 exercises

Unit 9. Techniques for Speeding Up Model Training [ Link to videos ]

• 9.1 Accelerated Model Training via Mixed-Precision Training
• 9.2 Multi-GPU Training Strategies
• 9.3 Deep Dive Into Data Parallelism
• 9.4 Compiling PyTorch Models
• 9.5 Increasing Batch Sizes to Increase Throughput
• Unit 9 exercises

Unit 10. The Finale: Our Next Steps After AI Model Training [ Link to videos ]

• 10.1 Trustworthy and Reliable Machine Learning
• 10.2 Fabric - Scaling PyTorch Model Training without Boilerplate Code
• 10.3 Designing Machine Learning Systems
• 10.4 Conclusion
• Unit 10 exercises