Skip to content
Glacius
ProbabilityConcept reference

Joint expectations

A finite joint expectation sums a function’s value at each ordered pair times that pair’s joint probability.

1 min read1 figureSources & further reading
On this page 8 sections
  1. Overview
  2. For a function of two random variables, keep each output beside its joint probability
  3. For any supplied function g, evaluate g(x,y) at each pair before weighting
  4. A closer look
  5. Key takeaway
  6. Further questions
  7. Sources & further reading
  8. Concept connections

01For a function of two random variables, keep each output beside its joint probability#

For a function of two random variables, keep each output beside its joint probability. Multiply those pairs of numbers, then add.

A joint table has rows X=0,1 and columns Y=0,1 with masses (0.1,0.2) and (0.3,0.4). For g(X,Y)=XY, only pair (1,1) has nonzero output, namely 1. Its contribution to E[XY] is 1 times 0.4, so the expectation is 0.4.A joint table has rows X=0,1 and columns Y=0,1 with masses (0.1,0.2) and (0.3,0.4). For g(X,Y)=XY, only pair (1,1) has nonzero output, namely 1. Its contribution to E[XY] is 1 times 0.4, so the expectation is 0.4.
Figure 1A joint table has rows X=0,1 and columns Y=0,1 with masses (0.1,0.2) and (0.3,0.4). For g(X,Y)=XY, only pair (1,1) has nonzero output, namely 1. Its contribution to E[XY] is 1 times 0.4, so the expectation is 0.4.
Link to this figure ↗Download SVGDownload PNG

With binary X,YX,Y and joint rows (0.1,0.2);(0.3,0.4)(0.1,0.2);(0.3,0.4), the product XYXY is zero at three pairs. At (1,1)(1,1), its value 1 carries probability 0.4.

1(0.4)=0.41(0.4)=0.4
Check your reasoning

Rows X=(0,2)X=(0,2); columns Y=(0,1)Y=(0,1). Joint rows: (0.2,0.1);(0.3,0.4)(0.2,0.1);(0.3,0.4). g(x,y)=xyg(x,y)=xy. Find E[g(X,Y)]E[g(X,Y)].

Show answer and explanation
0.8

2(0.4)=0.82(0.4)=0.8.

02For any supplied function g, evaluate g(x,y) at each pair before weighting#

For any supplied function gg, evaluate g(x,y)g(x,y) at each pair before weighting. The probabilities must sum to 1; the function values need not.

E[g(X,Y)]=∑x,yg(x,y)p(x,y)\begin{gathered}E[g(X,Y)]\\=\sum_{x,y}g(x,y)p(x,y)\end{gathered}
Check your reasoning

X rows; Y columns; order 0,1. Joint rows: (0.5,0);(0,0.5)(0.5,0);(0,0.5). g(x,y)=xyg(x,y)=xy. Fix separate means. Find E[XY]E[XY].

  1. A0.250.25
  2. B11
  3. C0.50.5
Show answer and explanation
0.50.5

1(0.5)=0.51(0.5)=0.5.

03A closer look#

For binary X,YX,Y and joint rows (0.1,0.2);(0.3,0.4)(0.1,0.2);(0.3,0.4), squared difference (X−Y)2(X-Y)^2 is 1 at the two off-diagonal pairs. Its expectation is 0.2+0.3=0.50.2+0.3=0.5. In general, applying a nonlinear function to the means gives a different result.

Check your reasoning

Cost is g. X rows; Y columns; order 0,1. Joint rows: (0.4,0.1);(0.2,0.3)(0.4,0.1);(0.2,0.3). g(x,y)=(x−y)2g(x,y)=(x-y)^2. Find E[g(X,Y)]E[g(X,Y)].

Show answer and explanation
0.3

0.1+0.2=0.30.1+0.2=0.3.

Key takeaway

Evaluate at each pair, weight by its mass, then add.

  • Compute an expectation of a function using a finite joint table.

Further questions

Can I always use g(E[X],E[Y])?
No. Averaging a nonlinear function can give a different result from applying it to the means. Use the original joint weights.

Sources & further reading

  1. [1]
  2. [2]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Joint expectations.” Math behind ML. /learn/p-joint-expectation