Linear-Form Gradients

01Each coefficient is a sensitivity#

A linear form $f(x)=a^Tx$ multiplies each input coordinate by a fixed coefficient and sums. Its gradient records how that scalar changes when each input coordinate changes independently.

For $a=(2,-3)$, the function is $f(x)=2x_1-3x_2$. Differentiating with respect to $x_1$ gives $2$; differentiating with respect to $x_2$ gives $-3$.

02The gradient is not the value#

At $x=(4,1)$, the function value is $8-3=5$. Its gradient is still $(2,-3)$. The value depends on the input; this gradient does not, because the coefficients are constant.

The column gradient has the same shape as the column input $x$. The expression $a^Tx$ is a scalar even though its gradient has several entries.

03Read a weighted score#

For a supplied score $s(w)=3w_1+w_2-2w_3$, the gradient with respect to the weights is $(3,1,-2)$. Here the feature values are the fixed coefficients. Variable names do not change the rule; the chosen differentiation variable does matter.

A constant offset adds no derivative. For $s(w)=3w_1+w_2-2w_3+7$, the gradient is unchanged. This follows from the constant derivative rule.

The coefficient vector must be constant with respect to the variable. If it also depends on that variable, its derivative contributes additional terms. Checking what is held fixed is part of applying the rule correctly.