Skip to content
Glacius
ProbabilityConcept reference

IID samples

IID observations are independent random variables with identical probability distributions.

1 min read1 figureSources & further reading
On this page 7 sections
  1. Overview
  2. IID means independent and identically distributed
  3. For two binary variables, compare their marginal probabilities of 1
  4. Independent draws can have different laws
  5. Key takeaway
  6. Sources & further reading
  7. Concept connections

01IID means independent and identically distributed#

IID means independent and identically distributed. Check both: each draw has the same probability law, and the draws are generated independently. Matching observed values prove neither condition.

Two fresh independent fair 0/1 draws are IID. If the second instead copies the first, each is still fair, but knowing the first fixes the second.

Independent fair binary X,Y put mass 1/4 in each of the four joint cells. A fair X copied to Y instead puts mass 1/2 on (0,0) and (1,1), zero on the off-diagonal cells. Both have fair binary marginals; only the first pair is independent.Independent fair binary X,Y put mass 1/4 in each of the four joint cells. A fair X copied to Y instead puts mass 1/2 on (0,0) and (1,1), zero on the off-diagonal cells. Both have fair binary marginals; only the first pair is independent.
Figure 1Independent fair binary X,Y put mass 1/4 in each of the four joint cells. A fair X copied to Y instead puts mass 1/2 on (0,0) and (1,1), zero on the off-diagonal cells. Both have fair binary marginals; only the first pair is independent.
Link to this figure ↗Download SVGDownload PNG
Check your reasoning

Two independent uniform draws with replacement from {red, blue}. Same law / independent?

  1. AYes / no.
  2. BNo / yes.
  3. CYes / yes (IID).
Show answer and explanation
Yes / yes (IID).

Both conditions hold.

02For two binary variables, compare their marginal probabilities of 1#

For two binary variables, compare their marginal probabilities of 1. Then check whether each joint mass equals the product of its marginal masses. Same marginals alone do not imply independence.

Check your reasoning

Binary X,Y. Joint masses: 00=0, 01=0.5, 10=0.5, 11=0. Same law / independent?

  1. AYes / yes (IID).
  2. BYes / no.
  3. CNo / yes.
Show answer and explanation
Yes / no.

P(11)=0 ≠ 0.5×0.5.

03Independent draws can have different laws#

Independent draws can have different laws. Sampling without replacement can preserve each draw’s marginal law while making later draws depend on earlier ones. Without a stated mechanism, independence may be unknown.

Check your reasoning

A fair 0/1 Z is copied: X=Z and Y=Z. “IID.” Same law / independent?

  1. AYes / no.
  2. BYes / yes (IID).
  3. CNo / yes.
Show answer and explanation
Yes / no.

Copying forces equality.

Key takeaway

Check the common law and independent generation separately. Neither equal marginals nor matching observed values proves both.

  • Check identical laws and independence separately.

Sources & further reading

  1. [1]
    Pishro-Nik, Introduction to Probability, 8.1.1 Random Sampling ↗Pishro-Nik, Introduction to Probability · Book

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “IID samples.” Math behind ML. /learn/p-iid