Layer Normalization

Formula: $\widehat{x_j}=\frac{x_j-{\text{micro}}_j}{\sqrt{{\sigma }_j^2+ϵ}}$

We also introduce 2 parameters:

• Gamma (Multiplicative)
• Beta (Additive)

This introduces some fluctuation in the data because having all data between 0 and 1 may be too restrictive for network. The network will learn to tune these 2 parameters to introduce fluctuation when necessary