Scalar chain rule

01Follow the values before the derivatives#

Suppose a computation first forms $u=2x+3$, then squares it to get $y=u^2$. To find how $y$ changes with $x$, we must account for both operations.

Call the inner function $g(x)=2x+3$ and the outer function $f(u)=u^2$. Their composition is $y=f(g(x))$. The inner function acts first, even though it is written inside the outer one.

At $x=1$, the intermediate value is $u=5$ and the output is $y=25$. These are the forward values. Keep them available: the derivative of the square depends on which value we are squaring.

The diagram follows this calculation and records the local rate of each operation.

Link to this figure ↗Download SVGDownload PNGCopy linkCopy embed

02Multiply the two local rates#

The first operation has rate $du/dx=2$. A small input change $h$ therefore changes $u$ by $2h$.

The square has rate $dy/du=2u$. At the intermediate value $u=5$, that rate is $10$. It turns the small change $2h$ into an output change of about $10(2h)=20h$.

The overall rate is $20$, obtained by multiplying the inner rate by the outer rate. The numbers $2$ and $10$ are rates; the forward values $5$ and $25$ are different quantities.

The derivative notation helps track what changes with respect to what. The rule is a statement about differentiable functions, not ordinary cancellation of independent numbers.

For a numerical check, increase $x$ from $1$ to $1.01$. Then $u=5.02$ and $y=25.2004$.

Our derivative predicted an output change of $20(0.01)=0.2$, giving $25.2$. The extra $0.0004$ is the quadratic term. Multiplying derivatives captures the first-order effect.

03Evaluate the outer derivative at the inner value#

For differentiable functions $g$ and $f$, the scalar chain rule is

A prime means derivative. The expression $f'(g(x))$ means evaluate the derivative of $f$ at the value $g(x)$. The outer function receives that intermediate value, so its derivative must be evaluated there too.

In our example, $f'(u)=2u$, so $f'(g(1))=2(5)=10$. Using $f'(1)=2$ would evaluate the outer rate at an input it never received.

This distinction becomes more consequential in long computations: each operation's derivative belongs at its own forward input.

04Work through a nonlinear inner function#

Consider $y=(x^2+1)^3$. Identify the inner expression $u=x^2+1$ and the outer cube $y=u^3$.

The inner derivative is $du/dx=2x$. The outer derivative is $dy/du=3u^2$. Substitute the inner expression into the outer derivative, then multiply:

Differentiating only the outer cube would leave out the factor $2x$. That would treat a change in $x$ as if it produced an equal change in $u$, which is generally false.

At $x=2$, the inner value is $5$, the outer rate is $75$, and the inner rate is $4$. The combined rate is $300$.

05Track signs and dependency paths#

For $y=(2-x)^2$, the inner derivative is $-1$, giving

At $x=0$, the derivative is $-4$. Increasing $x$ slightly reduces the inner value from $2$, which reduces its square. Losing the negative inner rate would predict the wrong direction.

An inner derivative of zero also matters: the first-order effect through that path is zero at that point. It does not necessarily mean the entire composition is constant.

For example, the derivative of $x^2$ vanishes at $x=0$, even though $x^2$ changes away from zero. Derivatives describe local first-order behavior.

In a chain of several scalar operations, multiply the local rates along the dependency path. If an input affects the output through several paths, each path contributes.

The multivariable chain rule adds those path contributions. Jacobian multiplication is the matrix form of this same operation: multiply along a path, then sum over intermediate coordinates.