Total expectation

01Average the group averages#

The law of total expectation recovers an overall mean by weighting conditional means according to how often their cases occur. The cases form a complete, disjoint partition of the population.

Let $\mu_y=E[X\mid Y=y]$ be the mean in case $y$. Weight each mean by its case probability:

One average happens inside each case. The outer average then combines the cases. Equal case weights represent an equally likely mixture. For unequal probabilities, use the actual weights even when equal averaging happens to give the same result.

02Unequal groups need unequal weights#

Suppose cases $a,b$ occur with probabilities $0.8,0.2$, and their conditional means are $1,6$. Their contributions to the overall mean are $0.8$ and $1.2$.

The unweighted average of the group means is $3.5$, which describes a different mixture. Small high-loss groups can matter greatly, but their contribution still uses their probability.

03Reconstruct the joint weights#

Substitute each conditional mean's weighted sum into the rule. A value $x$ in case $y$ receives weight $P(y)P(x\mid y)=P(x,y)$. Summing across cases recovers its marginal probability $P(x)$ and hence the original expectation.

In ML, averaging performance separately across groups is not enough to obtain a target population's expected loss. You also need that population's group proportions. Changing the proportions can change the overall mean even when every within-group mean stays fixed.