Skip to content
Glacius
ProbabilityConcept reference

Independent variables

Two discrete random variables are independent when every joint probability equals the product of the corresponding marginal probabilities.

1 min read1 figureSources & further reading
On this page 8 sections
  1. Overview
  2. Understand the idea
  3. Joint rows (0.2,0.3);(0.2,0.3) have row totals (0.5,0.5) and column totals (0.4,0.6)
  4. Equal-looking marginals do not establish independence
  5. Key takeaway
  6. Further questions
  7. Sources & further reading
  8. Concept connections

01Understand the idea#

Independence requires every joint cell to equal its row probability times its column probability.

Joint rows X=0,1 and columns Y=0,1 are (0.1,0.2) and (0.3,0.4). Row sum P(X=0)=0.3 and column sum P(Y=0)=0.4 have product 0.12, while the joint corner is 0.1. This mismatch disproves independence.Joint rows X=0,1 and columns Y=0,1 are (0.1,0.2) and (0.3,0.4). Row sum P(X=0)=0.3 and column sum P(Y=0)=0.4 have product 0.12, while the joint corner is 0.1. This mismatch disproves independence.
Figure 1Joint rows X=0,1 and columns Y=0,1 are (0.1,0.2) and (0.3,0.4). Row sum P(X=0)=0.3 and column sum P(Y=0)=0.4 have product 0.12, while the joint corner is 0.1. This mismatch disproves independence.
Link to this figure ↗Download SVGDownload PNG

For joint rows (0.1,0.2);(0.3,0.4)(0.1,0.2);(0.3,0.4), the top-left mass is 0.1. Its row and column totals give 0.3(0.4)=0.120.3(0.4)=0.12. These differ, so the variables are dependent.

p(x,y)=PX(x)PY(y)p(x,y)=P_X(x)P_Y(y)
Check your reasoning

X rows, Y cols; each 0,1. Mass rows: (0.15,0.1);(0.45,0.3)(0.15,0.1);(0.45,0.3). Independent?

  1. ADependent.
  2. BIndependent.
  3. CUndetermined.
Show answer and explanation
Independent.

All four cells factor.

02Joint rows (0.2,0.3);(0.2,0.3) have row totals (0.5,0.5) and column totals (0.4,0.6)#

Joint rows (0.2,0.3);(0.2,0.3)(0.2,0.3);(0.2,0.3) have row totals (0.5,0.5)(0.5,0.5) and column totals (0.4,0.6)(0.4,0.6). Multiplying them reproduces all four cells, so these variables are independent.

Check your reasoning

X rows, Y cols; each 0,1. Mass rows: (0.4,0.1);(0.1,0.4)(0.4,0.1);(0.1,0.4). Same marginals, so independent?

  1. AIndependent.
  2. BUndetermined.
  3. CDependent.
Show answer and explanation
Dependent.

Cell (0,0)(0,0): 0.4≠0.250.4\ne 0.25.

03Equal-looking marginals do not establish independence#

Equal-looking marginals do not establish independence. A single matching cell is also insufficient: inspect the complete table. These are supplied probability models, not estimates from observed samples.

Check your reasoning

X,Y: binary flags. X rows, Y cols; each 0,1. Mass rows: (0.1,0.4);(0.1,0.4)(0.1,0.4);(0.1,0.4). Independent?

  1. AIndependent.
  2. BDependent.
  3. CUndetermined.
Show answer and explanation
Independent.

All four cells factor.

Key takeaway

Every cell must factor; one mismatch is enough to fail.

  • Check factorization of a discrete joint distribution into its marginals.

Further questions

Can one matching cell prove independence?
No. Every pair must satisfy the product rule. One mismatched pair is enough to disprove it.

Sources & further reading

  1. [1]
  2. [2]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Independent variables.” Math behind ML. /learn/p-rv-independent