Cross Entropy Loss

$L=-\frac{1}{N}{\sum }_{i=1}^N\left[y_i\log ({\hat{y}}_i)+(1-y_i)\log (1-{\hat{y}}_i)\right]$

• Used in: Classification problems.
• Intuition: Measures the distance between the predicted probability distribution and the true distribution.
• Characteristics:
  • Strongly penalizes confident but wrong predictions.
  • Works naturally with probability outputs (sigmoid/softmax).