Quadratic forms

01A vector becomes a scalar cost#

A quadratic form takes a vector $x$, transforms it with a square matrix $A$, and dots the result with $x$. Its value is a scalar. Quadratic forms describe costs that depend on how coordinates interact.

For an $n$-entry column vector, the shapes are $(1\times n)(n\times n)(n\times1)$. The final shape is $1\times1$. Computing $Ax$ first makes the two operations clear.

02Transform, then dot#

Let $A$ have rows $(2,1),(1,3)$ and let $x=(1,2)$. The matrix-vector product is $Ax=(4,7)$. Dotting with $x$ gives $1(4)+2(7)=18$.

Expanding directly gives the same result: $2x_1^2+x_1x_2+x_2x_1+3x_2^2$. Both off-diagonal entries contribute. Counting just the diagonal would miss the cross-coordinate terms.

03Inspect the result#

A scalar result need not be positive. If $A$ has diagonal entries $1,-1$ and all other entries zero, then $x=(0,2)$ gives $q(x)=-4$. Classifying matrices whose forms are always nonnegative comes later.

If you replace $x$ by $cx$, the form is multiplied by $c^2$: the scale appears in each copy of the vector. Negating the whole vector therefore leaves the value unchanged.

Keep the intermediate vector $Ax$ separate from the final scalar $x^TAx$. This distinction also matters when differentiating the form: a scalar function can have a vector gradient.