Projection onto a subspace

01Projection onto a subspace fits a target using several allowed directions together#

Projection onto a subspace fits a target using several allowed directions together. When those directions are orthonormal, each dot product measures one independent coordinate, so we can fit their contributions separately and add them.

The target might be a list of measurements and the directions might be patterns your model can produce. The projected vector is the best fit inside their span. The residual is the part that these patterns cannot reproduce.

For orthonormal basis $q_1=(1,0,0)$, $q_2=(0,1,0)$, projecting $v=(2,3,4)$ retains $(2,3,0)$. Add the two line projections.

Link to this figure ↗Download SVGDownload PNGCopy linkCopy embed

For an orthonormal basis $q_1,q_2$, take coefficients $c_1=v\cdot q_1$ and $c_2=v\cdot q_2$. Each denominator $q_i\cdot q_i$ is $1$.

For more basis vectors, add one such term per vector.

With orthonormal directions $q_1=(1,0,0)$ and $q_2=(0,1,0)$, target $b=(2,3,4)$ has coordinates 2 and 3. Rebuild the projection as $2q_1+3q_2=(2,3,0)$. The residual $(0,0,4)$ is perpendicular to both directions. Reporting $(2,3)$ alone would report internal coordinates, not the three-entry projected vector.

02Use the supplied orthonormal basis u=(3/5,4/5,0), w=(0,0,1) and v=(5,0,2)#

Use the supplied orthonormal basis $u=(3/5,4/5,0)$, $w=(0,0,1)$ and $v=(5,0,2)$. The first coefficient is $5(3/5)+0(4/5)=3$. The second is $v\cdot w=2$.

For $u=(3/5,4/5,0)$ and $w=(0,0,1)$, coefficients $3,2$ give $3u=(9/5,12/5,0)$ and $2w=(0,0,2)$.

The pair $(3,2)$ contains weights. The projection has three coordinates.

03With v=(2,3,4) and basis (1,0,0),(0,1,0), the projection is p=(2,3,0)#

With $v=(2,3,4)$ and basis $(1,0,0),(0,1,0)$, the projection is $p=(2,3,0)$. The residual $v-p=(0,0,4)$ has dot product zero with both basis vectors.

A vector already in the span stays unchanged. A vector perpendicular to every basis vector projects to zero.

04The simple sum of dot-product contributions assumes orthonormal directions#

The simple sum of dot-product contributions assumes orthonormal directions. For a general set of independent columns, first obtain suitable orthonormal directions or solve the appropriate least-squares system. Otherwise overlapping contributions can be counted twice.