Skip to content
Glacius
StatisticsConcept reference

Centering data

Centering subtracts the training-sample mean of each feature from that feature’s values.

2 min read1 figureSources & further reading
On this page 9 sections
  1. Overview
  2. PCA looks for variation around a typical observation
  3. Each centered training column sums to zero
  4. Keep training means fixed for new data
  5. When transforming a new observation, reuse the means learned from the training set
  6. Key takeaway
  7. Further questions
  8. Sources & further reading
  9. Concept connections

01PCA looks for variation around a typical observation#

PCA looks for variation around a typical observation. Centering moves that typical observation to the origin: subtract each feature’s training mean from every value of that feature. Differences from the mean then describe variation, rather than the arbitrary location of the original zero.

Suppose two sensors have mean readings 10 and 100. A reading (12,97)(12,97) becomes (2,−3)(2,-3): the first is 2 above its typical value and the second is 3 below. Subtracting one grand average would mix the meanings of the sensors.

Centering subtracts one mean per feature. With observations as rows, compute the mean down each column. Each centered value keeps its original units.

Xc,ij=Xij−xˉjX_{c,ij}=X_{ij}-\bar x_j

Training rows (2,8)(2,8) and (6,12)(6,12) have means (4,10)(4,10). Subtract them from both rows: (−2,−2)(-2,-2) and (2,2)(2,2).

Training observations (2,8),(6,12) have feature mean (4,10). Subtracting that same vector moves them to (−2,−2),(2,2). Dashed displacement lines show centering; distances between observations are unchanged.Training observations (2,8),(6,12) have feature mean (4,10). Subtracting that same vector moves them to (−2,−2),(2,2). Dashed displacement lines show centering; distances between observations are unchanged.
Figure 1Training observations (2,8),(6,12) have feature mean (4,10). Subtracting that same vector moves them to (−2,−2),(2,2). Dashed displacement lines show centering; distances between observations are unchanged.
Link to this figure ↗Download SVGDownload PNG

Place observations in rows and features in columns. For rows (2,10)(2,10) and (4,14)(4,14), the column means are (3,12)(3,12). Subtract the same pair from both rows to obtain (−1,−2)(-1,-2) and (1,2)(1,2). Each centered column sums to zero. This is a useful check, not a reason to center each row separately.

Check your reasoning

Training observation rows: (2,6)(2,6); (4,10)(4,10). Give centered rows.

  1. A(−1,−2)(-1,-2); (1,2)(1,2)
  2. B(−2,2)(-2,2); (−3,3)(-3,3)
  3. C(5,14)(5,14); (7,18)(7,18)
Show answer and explanation
(−1,−2)(-1,-2); (1,2)(1,2)

Subtract feature means (3,8)(3,8).

02Each centered training column sums to zero#

Each centered training column sums to zero. Subtracting a separate mean across each row answers a different question. Dividing by standard deviation is scaling; it is not part of centering.

Check your reasoning

Training observation rows: (1,5)(1,5); (3,9)(3,9). Claimed centered rows: (−2,2)(-2,2); (−3,3)(-3,3). Repair.

  1. A(−2,2)(-2,2); (−3,3)(-3,3)
  2. B(−1,−2)(-1,-2); (1,2)(1,2)
  3. C(3,12)(3,12); (5,16)(5,16)
Show answer and explanation
(−1,−2)(-1,-2); (1,2)(1,2)

Subtract feature means (2,7)(2,7).

03Keep training means fixed for new data#

Keep training means fixed for new data. Training rows (1,7)(1,7) and (5,9)(5,9) give means (3,8)(3,8). A new row (7,6)(7,6) becomes (4,−2)(4,-2). Do not let that new row change the training means.

Check your reasoning

Training rows: (0,4)(0,4); (4,8)(4,8). New row (5,3)(5,3). Center the new row.

  1. A(1,−1)(1,-1)
  2. B(2,−2)(2,-2)
  3. C(3,−3)(3,-3)
Show answer and explanation
(3,−3)(3,-3)

Use training means (2,6)(2,6).

04When transforming a new observation, reuse the means learned from the training set#

When transforming a new observation, reuse the means learned from the training set. Recomputing them on the test set changes the transformation and lets evaluation data influence the procedure. Centering subtracts means; standardization additionally rescales features and can change PCA directions.

Key takeaway

Center columns with training means. Reuse those means for new data; keep the original units.

  • Center each data feature using training-sample means.

Further questions

Does centering make every feature equally variable?
No. It shifts the means to zero on the training data but does not divide by a standard deviation or equalize feature variances.

Sources & further reading

  1. [1]
  2. [2]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Centering data.” Math behind ML. /learn/s-pca-center