# Week 3: Neural Networks

From the McCulloch-Pitts neuron to trained deep networks: perceptrons, MLPs, backpropagation, gradient descent and its variants, regularization, and better training.

## Lectures

1. [The McCulloch-Pitts Neuron and the Perceptron](<../notebooks/Week 3/L01_MP_Neuron_and_Perceptron.ipynb>)
2. [The XOR Conundrum, MLPs, and Activation Functions](<../notebooks/Week 3/L02_XOR_MLP_and_Activations.ipynb>)
3. [Feedforward Networks and Gradient Descent](<../notebooks/Week 3/L03_Feedforward_and_Gradient_Descent.ipynb>)
4. [Backpropagation, Built by Hand and Verified](<../notebooks/Week 3/L04_Backpropagation.ipynb>)
5. [Momentum and Nesterov Accelerated Gradient](<../notebooks/Week 3/L05_Momentum_and_Nesterov.ipynb>)
6. [SGD and Adaptive Optimizers, Built From Scratch](<../notebooks/Week 3/L06_SGD_and_Adaptive_Optimizers.ipynb>)
7. [Regularization and L2 Weight Decay](<../notebooks/Week 3/L07_Regularization_and_L2.ipynb>)
8. [Early Stopping, Data Augmentation, Noise, Ensembles, and Dropout](<../notebooks/Week 3/L08_EarlyStopping_Augmentation_Dropout.ipynb>)
9. [Activation Functions In Depth](<../notebooks/Week 3/L09_Activation_Functions_In_Depth.ipynb>)
10. [Weight Initialization and Batch Normalization](<../notebooks/Week 3/L10_Weight_Init_and_BatchNorm.ipynb>)