Batch Normalization

Batch Normalization is applied to layers of a Neural Networks where the output from the activation function is scaled

Here are the steps for BatchNorm

1. $z=\frac{x-m}{s}$
2. $output=(z\ast g)+b$

• m - mean
• s - standard deviation
• g - arbitrary parameter
• b - arbitrary parameter

m, s, g, and b are all trainable parameters so they’ll improve with more training