Skip to content
Glacius
StatisticsConcept reference

Effect sizes

An effect size expresses a difference in a stated scale and unit.

2 min read1 figureSources & further reading
On this page 7 sections
  1. Overview
  2. An effect size describes how much a quantity differs
  3. With samples A = 2, 4 and B = 5, 7, the group means are 3 and 6
  4. A report may call an effect statistically detectable
  5. Key takeaway
  6. Sources & further reading
  7. Concept connections

01An effect size describes how much a quantity differs#

An effect size describes how much a quantity differs. Here, define the effect as mean B minus mean A. Keep the measurement unit. A negative value means B is lower; zero means the means match.

Mean A is 30 ms and mean B is 22 ms. Subtract in the declared order: 22−30=−822-30=-8 ms. B is eight milliseconds lower.

On a common linear millisecond scale from 20 to 32, mean A is 30 and mean B is 22. A left-pointing arrow from 30 to 22 shows B minus A equals −8 milliseconds: B is eight milliseconds lower. This is an absolute difference, not a percentage.On a common linear millisecond scale from 20 to 32, mean A is 30 and mean B is 22. A left-pointing arrow from 30 to 22 shows B minus A equals −8 milliseconds: B is eight milliseconds lower. This is an absolute difference, not a percentage.
Figure 1On a common linear millisecond scale from 20 to 32, mean A is 30 and mean B is 22. A left-pointing arrow from 30 to 22 shows B minus A equals −8 milliseconds: B is eight milliseconds lower. This is an absolute difference, not a percentage.
Link to this figure ↗Download SVGDownload PNG
Check your reasoning

Means in ms: A=40, B=34. Effect δ=mean B−mean A. Find δ in ms.

Show answer and explanation
-6

34−40=-6 ms.

02With samples A = 2, 4 and B = 5, 7, the group means are 3 and 6#

With samples A = 2, 4 and B = 5, 7, the group means are 3 and 6. Their difference is 6−3=36-3=3 in the original unit. A difference of means alone does not establish that changing groups caused it.

Practical importance needs a context. Suppose an increase of at least 4 points matters. A supplied confidence interval [1,7][1,7] includes effects both below and above 4. Its positive values alone do not settle that practical threshold.

Check your reasoning

Effect δ in points. Supplied confidence interval [2,6]. Meaningful increase: δ≥4. Which interval values meet it?

  1. AAll interval values.
  2. BSome but not all.
  3. CNo interval values.
Show answer and explanation
Some but not all.

Compare [2,6] with 4.

03A report may call an effect statistically detectable#

A report may call an effect statistically detectable. That label does not state its size or importance. Compare the estimate and uncertainty with a threshold chosen for the application. A point estimate meeting a threshold is not a guarantee about the unknown effect.

Check your reasoning

Means A=6, B=8 points. δ=B−A. Meaningful: δ≥5. “Detectable, so meaningful.” Repair.

  1. A2 points; meets.
  2. B−2 points; below.
  3. C2 points; below.
Show answer and explanation
2 points; below.

8−6=2, below 5.

Key takeaway

Subtract in the declared order, keep the unit and compare the magnitude and uncertainty with a meaningful threshold.

  • Interpret the signed difference in its original unit.

Sources & further reading

  1. [1]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Effect sizes.” Math behind ML. /learn/s-effect