Skip to content
Glacius
ProbabilityConcept reference

Variance of sums

The variance of a sum equals the two variances plus twice their covariance, when these second moments are finite.

2 min read1 figureSources & further reading
On this page 9 sections
  1. Overview
  2. The variability of a total depends on how its parts move together
  3. Negative covariance reduces the variance of the sum
  4. When covariance is zero, the two variances add
  5. This matters when averaging correlated measurement errors or combining model errors
  6. Key takeaway
  7. Further questions
  8. Sources & further reading
  9. Concept connections

01The variability of a total depends on how its parts move together#

The variability of a total depends on how its parts move together. Two individually noisy quantities can reinforce each other or partly cancel. Variance therefore needs a covariance term unless a stated assumption makes that term zero.

The variance of a sum includes two cross terms. Covariance is symmetric, so both contribute the same value.

Given Var(X)=4, Var(Y)=9 and Cov(X,Y)=2, the four expansion terms are 4,2,2,9. The two equal covariance contributions total 4; the overall variance is 17. This positive-contribution area example does not assert that covariance is always positive.Given Var(X)=4, Var(Y)=9 and Cov(X,Y)=2, the four expansion terms are 4,2,2,9. The two equal covariance contributions total 4; the overall variance is 17. This positive-contribution area example does not assert that covariance is always positive.
Figure 1Given Var(X)=4, Var(Y)=9 and Cov(X,Y)=2, the four expansion terms are 4,2,2,9. The two equal covariance contributions total 4; the overall variance is 17. This positive-contribution area example does not assert that covariance is always positive.
Link to this figure ↗Download SVGDownload PNG

With variances 4 and 9 and covariance 2, adding twice the signed covariance gives 4+9+2(2)=174+9+2(2)=17.

Var⁡(X+Y)=Var⁡(X)+Var⁡(Y)+2Cov⁡(X,Y)\begin{gathered}\operatorname{Var}(X+Y)\\=\operatorname{Var}(X)+\operatorname{Var}(Y)\\+2\operatorname{Cov}(X,Y)\end{gathered}

If Var⁡(X)=4\operatorname{Var}(X)=4, Var⁡(Y)=9\operatorname{Var}(Y)=9, and Cov⁡(X,Y)=−2\operatorname{Cov}(X,Y)=-2, then Var⁡(X+Y)=4+9+2(−2)=9\operatorname{Var}(X+Y)=4+9+2(-2)=9. Ignoring covariance would give 13. The negative covariance reflects deviations that partly offset each other in the sum.

Check your reasoning

Var⁡(X)=4\operatorname{Var}(X)=4, Var⁡(Y)=9\operatorname{Var}(Y)=9, Cov⁡(X,Y)=−2\operatorname{Cov}(X,Y)=-2. Find Var⁡(X+Y)\operatorname{Var}(X+Y).

Show answer and explanation
9

4+9+2(−2)=94+9+2(-2)=9.

02Negative covariance reduces the variance of the sum#

Negative covariance reduces the variance of the sum. With variances 1 and 1 and covariance −1-1, the total variance is 1+1−2=01+1-2=0. In that case the sum is constant with probability 1.

Check your reasoning

Var⁡(X)=1\operatorname{Var}(X)=1, Var⁡(Y)=4\operatorname{Var}(Y)=4, Cov⁡(X,Y)=1\operatorname{Cov}(X,Y)=1. Repair including covariance once. Find Var⁡(X+Y)\operatorname{Var}(X+Y).

  1. A55
  2. B66
  3. C77
Show answer and explanation
77

1+4+2(1)=71+4+2(1)=7.

03When covariance is zero, the two variances add#

When covariance is zero, the two variances add. Independence is sufficient for zero covariance when these variances are finite, but zero covariance alone does not prove independence.

Check your reasoning

Total delay is X+Y. Var⁡(X)=9\operatorname{Var}(X)=9, Var⁡(Y)=16\operatorname{Var}(Y)=16, Cov⁡(X,Y)=3\operatorname{Cov}(X,Y)=3. Find Var⁡(X+Y)\operatorname{Var}(X+Y).

Show answer and explanation
31

9+16+2(3)=319+16+2(3)=31.

04This matters when averaging correlated measurement errors or combining model errors#

This matters when averaging correlated measurement errors or combining model errors. More observations do not automatically deliver the independent-sample variance reduction if the errors tend to move together. Always check which covariance or independence assumption the task supplies.

Key takeaway

Add both variances and twice the signed covariance.

  • Compute variance of a sum with a supplied covariance.

Further questions

Can negative covariance make the total variance negative?
No. A valid covariance and two valid variances must produce a nonnegative result. Negative covariance can reduce the variance to zero.

Sources & further reading

  1. [1]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Variance of sums.” Math behind ML. /learn/p-sum-variance