Week 3: Neural Networks

Contents

Week 3: Neural Networks#

From the McCulloch-Pitts neuron to trained deep networks: perceptrons, MLPs, backpropagation, gradient descent and its variants, regularization, and better training.

Lectures#

  1. The McCulloch-Pitts Neuron and the Perceptron

  2. The XOR Conundrum, MLPs, and Activation Functions

  3. Feedforward Networks and Gradient Descent

  4. Backpropagation, Built by Hand and Verified

  5. Momentum and Nesterov Accelerated Gradient

  6. SGD and Adaptive Optimizers, Built From Scratch

  7. Regularization and L2 Weight Decay

  8. Early Stopping, Data Augmentation, Noise, Ensembles, and Dropout

  9. Activation Functions In Depth

  10. Weight Initialization and Batch Normalization

Contents