Singular values

01Understand the idea#

A singular value tells you how strongly a matrix stretches one of its special input directions. In $Av_i=\sigma_i u_i$, both direction vectors have length 1. The output therefore has length $\sigma_i$. These scales are nonnegative; a negative coordinate elsewhere in the factorization is not a negative singular value.

Picture measurements that respond strongly to one input pattern and not at all to another. A zero singular value means changing the input along that second direction changes none of the outputs. Counting directions that survive tells us how many independent directions the matrix can produce: its rank.

A positive singular scale preserves its direction; a zero scale removes it. Complete scales $(4,2,0)$ preserve two independent directions, so the rank is 2.

Link to this figure ↗Download SVGDownload PNGCopy linkCopy embed

Count nonzero values, not their magnitudes. For a $4\times3$ matrix with complete exact scales $(5,5,0)$, both copies of 5 count.

For complete exact singular values $(4,2,0)$, ask about each scale separately: 4 preserves a direction, 2 preserves another, and 0 removes one. The rank is 2. Neither the sum 6 nor the largest value 4 counts directions. Check that the list has $\min(m,n)$ entries for an $m\times n$ matrix.

02Every exact positive scale counts, however small#

Every exact positive scale counts, however small. Scales $(2,0.001,0)$ give rank 2. Numerical software may choose a tolerance, but that is a separate decision; these tasks use exact values.

03Suppose a channel edit changes complete singular scales (6,2,0) to (6,0,0)#

Suppose a channel edit changes complete singular scales $(6,2,0)$ to $(6,0,0)$. One active direction is removed, so rank falls from 2 to 1. Changing 6 to another positive value would preserve the count.

04With measured data, a tiny positive scale may be mostly noise#

With measured data, a tiny positive scale may be mostly noise. A numerical-rank calculation therefore needs a stated tolerance and a scale convention. These exercises supply exact values and ask exact rank; every positive value counts. This distinction matters before declaring that a dataset contains a redundant feature.

Further questions