Skip to content
Glacius
ProbabilityConcept reference

Linearity of expectation

Expectation preserves finite linear combinations of random variables and constants.

1 min read1 figureSources & further reading
On this page 7 sections
  1. Overview
  2. Expectations add even when the variables are dependent
  3. Two counts may respond to the same underlying event
  4. This rule applies to linear combinations
  5. Key takeaway
  6. Sources & further reading
  7. Concept connections

01Expectations add even when the variables are dependent#

Expectations add even when the variables are dependent. Fixed coefficients and added constants also pass through the expectation.

E[aX+bY+c]=aE[X]+bE[Y]+c\begin{gathered}E[aX+bY+c]\\=aE[X]+bE[Y]+c\end{gathered}

For E[X]=2E[X]=2 and E[Y]=5E[Y]=5, the mean of 3X−Y3X-Y is 3(2)−5=13(2)-5=1.

Given E[X]=2 and E[Y]=5, the signed mean contributions are 6 and −5, summing to 1. No joint distribution or independence assumption is introduced.Given E[X]=2 and E[Y]=5, the signed mean contributions are 6 and −5, summing to 1. No joint distribution or independence assumption is introduced.
Figure 1Given E[X]=2 and E[Y]=5, the signed mean contributions are 6 and −5, summing to 1. No joint distribution or independence assumption is introduced.
Link to this figure ↗Download SVGDownload PNG
Check your reasoning

E[X]=3E[X]=3, E[Y]=2E[Y]=2. Find E[2X−Y+1]E[2X-Y+1].

Show answer and explanation
5

Linearity gives 2(3)+(−1)(2)+(1)=52(3)+(-1)(2)+(1)=5. Independence is unnecessary.

02Two counts may respond to the same underlying event#

Two counts may respond to the same underlying event. That dependence does not prevent you from finding the expected total using their individual means.

A fixed offset contributes itself. A coefficient multiplies the mean of its own variable.

Check your reasoning

Counts X,YX,Y may share a cause. E[X]=5E[X]=5, E[Y]=4E[Y]=4. Score S=2X+YS=2X+Y. Find E[S]E[S].

Show answer and explanation
14

Linearity gives 2(5)+4=142(5)+4=14. Independence is unnecessary.

03This rule applies to linear combinations#

This rule applies to linear combinations. It does not say that E[XY]=E[X]E[Y]E[XY]=E[X]E[Y], or that expectation can move through a square.

For the requested sum or affine score, dependence creates no extra term.

Check your reasoning

X,YX,Y are dependent, with E[X]=4E[X]=4 and E[Y]=3E[Y]=3. A learner says independence is needed. Find E[X−Y+2]E[X-Y+2].

Show answer and explanation
3

Linearity gives 1(4)+(−1)(3)+(2)=31(4)+(-1)(3)+(2)=3. Independence is unnecessary.

Key takeaway

Apply each coefficient to its mean, add the constant, and keep going even when variables are dependent.

  • Compute the expectation of a linear combination without assuming independence.

Sources & further reading

  1. [1]
    Pishro-Nik, Introduction to Probability: Expectation ↗Pishro-Nik: Introduction to Probability · Article

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Linearity of expectation.” Math behind ML. /learn/p-linearity