Quadratic-Form Gradients

01The variable occurs twice#

In $q(x)=x^TAx$, the input vector occurs on both sides of the fixed matrix. Changing one coordinate can affect terms through either occurrence. The gradient must include both contributions.

Write $q(x)=\sum_i\sum_j x_iA_{ij}x_j$. Differentiating with respect to $x_k$ collects the terms where $i=k$ and those where $j=k$:

Those sums are the $k$th entries of $Ax$ and $A^Tx$.

02Use a nonsymmetric matrix#

Let $A$ have rows $(1,2),(0,3)$ and $x=(2,1)$. Then $Ax=(4,3)$ and $A^Tx=(2,7)$. Adding gives the gradient $(6,10)$.

Expanding gives $q(x)=x_1^2+2x_1x_2+3x_2^2$. Its partial derivatives are $2x_1+2x_2$ and $2x_1+6x_2$, again giving $(6,10)$ at $(2,1)$.

03When the shorter rule works#

If $A=A^T$, the two contributions match and the gradient becomes $2Ax$. Without symmetry, that shortcut can fail: in the worked example $2Ax=(8,6)$, which is incorrect.

A leading factor of one-half halves the gradient. Keep the fixed matrix and the differentiating variable explicit. This lesson computes the gradient; deciding whether the quadratic has a minimum requires additional structure.