Expected Value

Expected value, also called mathematical expectation, is the weighted average value of a Random Variable under a Probability Distribution.

It is written with the expectation operator:

Read this as “the expected value of ” or “the expectation of “.

Intuition §

Expected value is the long-run average outcome if the same random process is repeated many times.

It is not necessarily the value that is most likely to happen. For example, the expected value of a fair six-sided die is:

But rolling a is impossible. The expected value is the average over repeated rolls.

Discrete random variables §

For a discrete random variable , expectation is computed by summing over each possible value, weighted by its probability:

Example:

Then:

Continuous random variables §

For a continuous random variable, expectation is computed with an integral instead of a sum:

where is the probability density function.

Expectation of a function §

Expectation can also be taken over a function of a random variable:

For continuous variables:

This is useful because many objectives in machine learning are written as expectations over losses, rewards, or log probabilities.

Subscript notation §

Sometimes the distribution is written under the expectation symbol:

This means:

1. sample from the distribution
2. compute
3. average the result over many samples

For example:

means “the expected return when trajectories are sampled from policy .”

This notation appears often in Policy Gradient:

Conditional expectation §

Expectation can be conditioned on some information:

Read this as “the expected value of given .”

For example, in reinforcement learning:

This means the value of a state is the expected future return given that the agent is currently in state .

Linearity of expectation §

Expectation is linear:

This is true even if and are not independent.

Useful special cases:

Sample estimate §

In practice, expectations are often estimated by averaging samples:

where each is sampled from .

This is why many ML and RL objectives use expectations in theory but averages over minibatches or sampled trajectories in code.

Common readings §

Notation	Read as
	expected value of
	expected value of when is sampled from
	expected value of a function of
	expected value of given
	empirical/sample average estimate of expectation

Summary §

Expectation notation means taking an average with respect to a probability distribution.

The key idea is:

In ML and RL, usually means “average this quantity over samples from some data distribution, model, policy, or environment.”