Skip to content
Glacius
StatisticsConcept reference

Design matrices

A design matrix stores observations as rows and supplied features as columns; an intercept is represented by a column of ones.

1 min read1 figureSources & further reading
On this page 8 sections
  1. Overview
  2. Put one observation on each row
  3. Keep the declared column order
  4. Evaluate each supplied feature before storing it
  5. Key takeaway
  6. Further questions
  7. Sources & further reading
  8. Concept connections

01Put one observation on each row#

Put one observation on each row. For predictor values 2,−1,02,-1,0, the first column is always 11 and the second holds the predictor.

Three observations have predictor x values2,-1,0. Columns are intercept1 and x, in that order. The design rows are(1,2),(1,-1),(1,0).Three observations have predictor x values2,-1,0. Columns are intercept1 and x, in that order. The design rows are(1,2),(1,-1),(1,0).
Figure 1Three observations have predictor x values2,-1,0. Columns are intercept1 and x, in that order. The design rows are(1,2),(1,-1),(1,0).
Link to this figure ↗Download SVGDownload PNG

Multiplying an observation row by the coefficient vector produces its mean prediction. The leading 11 carries the intercept unchanged.

(1,x)(β0β1)=β0+xβ1\begin{gathered}(1,x)\begin{pmatrix}\beta_0\\\beta_1\end{pmatrix}\\=\beta_0+x\beta_1\end{gathered}
Check your reasoning

Observation xx values (1,−2)(1,-2). Columns (1,x)(1,x). Give design rows.

  1. A(0,1)(0,1); (0,−2)(0,-2)
  2. B(1,1)(1,1); (1,−2)(1,-2)
  3. C(1,1)(1,1); (−2,1)(-2,1)
Show answer and explanation
(1,1)(1,1); (1,−2)(1,-2)

Rows: (1,1)(1,1); (1,−2)(1,-2).

02Keep the declared column order#

Keep the declared column order. With columns (1,x,z)(1,x,z) and observed (x,z)=(2,−1)(x,z)=(2,-1), the row is (1,2,−1)(1,2,-1). An intercept zero would erase its coefficient.

Check your reasoning

Rows (x,z)(x,z): (3,0)(3,0); (−1,2)(-1,2). Columns (1,x,z)(1,x,z). Repair zero intercepts.

  1. A(1,3,0)(1,3,0); (1,−1,2)(1,-1,2)
  2. B(0,3,0)(0,3,0); (0,−1,2)(0,-1,2)
  3. C(0,1,3)(0,1,3); (2,1,−1)(2,1,-1)
Show answer and explanation
(1,3,0)(1,3,0); (1,−1,2)(1,-1,2)

Intercept column: all ones.

03Evaluate each supplied feature before storing it#

Evaluate each supplied feature before storing it. If h(x)=x2h(x)=x^2, then x=−2x=-2 produces row (1,4)(1,4) in columns (1,h)(1,h). Keep the intercept column at 11.

Check your reasoning

Record xx values (−3,0,2)(-3,0,2). Columns (1,h)(1,h); h=x2h=x^2. Give design rows.

  1. A(1,−3)(1,-3); (1,0)(1,0); (1,2)(1,2)
  2. B(0,9)(0,9); (0,0)(0,0); (0,4)(0,4)
  3. C(1,9)(1,9); (1,0)(1,0); (1,4)(1,4)
Show answer and explanation
(1,9)(1,9); (1,0)(1,0); (1,4)(1,4)

h=(9,0,4)h=(9,0,4); prepend 11.

Key takeaway

Use one row per observation, evaluate the declared features, and preserve the column order. The intercept column is all ones.

  • Encode supplied predictors and an intercept as a design matrix.

Further questions

How can a supplied QR factorization solve for the coefficient vector?
For a full-column-rank design X=QR, compute Q-transpose y and solve the upper-triangular system R beta=Q-transpose y. Preserve the design’s coefficient order.

Sources & further reading

  1. [1]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Design matrices.” Math behind ML. /learn/s-design