Skip to content
Glacius
StatisticsConcept reference

Sampling distributions

A sampling distribution is the probability distribution of an estimator under a fixed sampling scheme.

2 min read1 figureSources & further reading
On this page 8 sections
  1. Overview
  2. Understand the idea
  3. For a largest-reading rule, record the largest value in each new sample
  4. Some tasks list unequal sample probabilities
  5. Uncertainty intervals and standard errors concern this across-sample variation
  6. Key takeaway
  7. Sources & further reading
  8. Concept connections

01Understand the idea#

A sampling distribution describes the values a statistic would take across repeated samples generated by the same sampling process. It is a distribution of estimates, not a histogram of the original observations in one sample.

Imagine repeatedly drawing the same number of records and computing a mean each time. Each sample contributes one mean to the sampling distribution. Its spread measures how much the estimate would change if the sample were collected again.

A sampling distribution describes the outputs of an estimator across repeated samples. Keep the sampling scheme, sample size and rule fixed. Record one output per sample, rather than pooling its raw observations.

Suppose (0,0),(0,2),(2,0),(2,2)(0,0),(0,2),(2,0),(2,2) each have probability 1/41/4. For T=(x+y)/2T=(x+y)/2, their outputs are 0,1,1,20,1,1,2.

Four equally likely ordered samples (0,0),(0,2),(2,0),(2,2) have sample means 0,1,1,2. Two distinct samples produce mean 1, so the estimator law has masses 1/4,1/2,1/4. The horizontal axis is the sample-mean statistic, not individual data.Four equally likely ordered samples (0,0),(0,2),(2,0),(2,2) have sample means 0,1,1,2. Two distinct samples produce mean 1, so the estimator law has masses 1/4,1/2,1/4. The horizontal axis is the sample-mean statistic, not individual data.
Figure 1Four equally likely ordered samples (0,0),(0,2),(2,0),(2,2) have sample means 0,1,1,2. Two distinct samples produce mean 1, so the estimator law has masses 1/4,1/2,1/4. The horizontal axis is the sample-mean statistic, not individual data.
Link to this figure ↗Download SVGDownload PNG

Take two independent draws from a population equally likely to return 0 or 2. The ordered pairs (0,0),(0,2),(2,0),(2,2)(0,0),(0,2),(2,0),(2,2) each have probability 1/41/4. Their means are 0,1,1,20,1,1,2. Combine samples with the same mean: the probabilities at 0, 1, 2 are 1/4,1/2,1/41/4,1/2,1/4.

Both middle samples give T=1T=1. Add their probabilities: 1/4+1/4=1/21/4+1/4=1/2. Thus the masses at 0,1,20,1,2 are 1/4,1/2,1/41/4,1/2,1/4. Distinct outputs need not be equally likely.

Check your reasoning

All pairs (x,y): (0,0):1/4; (0,4):1/4; (4,0):1/4; (4,4):1/4. T=max(x,y). Masses at 0,4?

  1. A(1/4, 3/4)
  2. B(1/2, 1/2)
  3. C(3/4, 1/4)
Show answer and explanation
(1/4, 3/4)

P(T=0)=1/4; P(T=4)=3/4.

02For a largest-reading rule, record the largest value in each new sample#

For a largest-reading rule, record the largest value in each new sample. For a count rule, record that count. A histogram of all the individual readings describes a different variable.

Check your reasoning

Same random scheme: 5 readings. Rule: count readings above 10. Record what across samples?

  1. AAll raw readings.
  2. BEach sample’s count.
  3. COne fixed output.
Show answer and explanation
Each sample’s count.

Keep the count.

03Some tasks list unequal sample probabilities#

Some tasks list unequal sample probabilities. Use those masses when combining outputs. The notation (x,y):p(x,y):p means that complete pair has probability pp. The displayed list is exhaustive.

Check your reasoning

All pairs (x,y): (0,0):1/4; (0,2):1/2; (2,2):1/4. T=x+y. Masses at 0,2,4? “Distinct outputs get equal masses.”

  1. A(1/3, 1/3, 1/3)
  2. B(1/2, 0, 1/2)
  3. C(1/4, 1/2, 1/4)
Show answer and explanation
(1/4, 1/2, 1/4)

Masses: 1/4, 1/2, 1/4.

04Uncertainty intervals and standard errors concern this across-sample variation#

Uncertainty intervals and standard errors concern this across-sample variation. Observing many different values within one dataset is a different source of spread. Keep the sampling rule and sample size fixed when describing one sampling distribution.

Key takeaway

Evaluate the same rule on each possible sample and combine the probabilities of equal outputs.

  • Map repeated samples to estimator-output probabilities.

Sources & further reading

  1. [1]
  2. [2]
    Pishro-Nik, Introduction to Probability, 8.1.1 Random Sampling ↗Pishro-Nik, Introduction to Probability · Book

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Sampling distributions.” Math behind ML. /learn/s-sampling-distribution