CalculusConcept reference

Scalar chain rule

The derivative of a composition is the outer derivative evaluated at the inner output, times the inner derivative.

2 min read1 figureSources & further reading

On this page 8 sections

Overview
A model often calculates an intermediate quantity and then uses it in another calculation
For y=(x^2+1)^3, the outer cube has derivative 3u^2 at u=x^2+1
A negative inner rate can reverse the final sign
A prediction followed by a loss is a composition
Key takeaway
Sources & further reading
Concept connections

01A model often calculates an intermediate quantity and then uses it in another calculation#

A model often calculates an intermediate quantity and then uses it in another calculation. The chain rule finds how a small input change travels through both stages. The final sensitivity depends on the rate of each stage, evaluated at the value that stage actually receives.

For $y=f(g(x))$ , the inner output becomes the outer input. Multiply the outer rate at that intermediate value by the inner rate.

y\prime=f\prime(g(x))g\prime(x)

Let $g(x)=2x+3$ and $f(u)=u^2$ . At $x=1$ , the intermediate value is $u=5$ . The outer derivative is $2u$ , so its rate there is $10$ .

Figure 1Schematic composition, with no metric encoding. Input x=1 passes through g(x)=2x+3 to intermediate u=5, then f(u)=u² gives y=25. Inner local rate is 2 at x=1; outer local rate is 10 at u=5. End-to-end derivative is 10 times 2, or 20.

For $y=(2x+3)^2$ at $x=1$ , write $u=2x+3$ . The forward calculation gives $u=5$ . The inner rate is $du/dx=2$ ; the square’s rate at $u=5$ is $dy/du=10$ . Multiply to get $dy/dx=20$ . Using $2x=2$ for the outer rate would evaluate the square at the wrong input.

Near $x=1$ , the inner output changes about $2$ units per unit of $x$ . Near $u=5$ , the outer output changes about $10$ units per unit of $u$ . The combined rate is $10\times2=20$ .

The outer rate must be evaluated where the inner function actually lands.

Check your reasoning

$y=f(g(x))$ . At $x=2$ , $g(2)=7$ and $g\prime(2)=3$ . Also, $f\prime(7)=4$ and $f\prime(2)=5$ . Find $y\prime$ at this input.

Show answer and explanation

Use the outer rate at $7$ , then multiply: $4(3)=12$ .

02For y=(x^2+1)^3, the outer cube has derivative 3u^2 at u=x^2+1#

For $y=(x^2+1)^3$ , the outer cube has derivative $3u^2$ at $u=x^2+1$ . The inner derivative is $2x$ .

y\prime=3(x^2+1)^2(2x)

The outer factor alone is an incomplete derivative.

Check your reasoning

$y=(2x+1)^{3}$ . Find $y\prime(x)$ .

A $12x^{2}+12x+3$
B $24x^{2}+24x+6$
C $3x^{2}$

Show answer and explanation

$24x^{2}+24x+6$

Outer rate: $3(2x+1)^{2}$ . Inner rate: $2$ . Multiply them.

03A negative inner rate can reverse the final sign#

A negative inner rate can reverse the final sign. For $y=(2-x)^2$ , the inner derivative is $-1$ .

y\prime=2(2-x)(-1)

A zero inner rate makes the combined rate zero when the outer derivative exists at the intermediate value. Both stages must be differentiable at their respective inputs.

Check your reasoning

$y=f(g(x))$ . At $x=-1$ , $g(-1)=4$ and $g\prime(-1)=-2$ . Also, $f\prime(4)=6$ and $f\prime(-1)=3$ . Claim: use $f\prime(-1)$ . Repair $y\prime(-1)$ .

Show answer and explanation

-12

Use $f\prime(4)$ : $6(-2)=-12$ .

04A prediction followed by a loss is a composition#

A prediction followed by a loss is a composition. The chain rule tells training software how a weight affects the loss through its effect on the prediction. Keeping the intermediate value and the local derivative separate is the habit used in backpropagation.

Key takeaway

Evaluate the outer rate at the inner output, then multiply by the inner rate.

Differentiate a composition of two scalar functions.

Sources & further reading

[1]
OpenStax Calculus Volume 1, 3.6: The chain rule ↗OpenStax Calculus Volume 1 · Book

01A model often calculates an intermediate quantity and then uses it in another calculation#

02For y=(x^2+1)^3, the outer cube has derivative 3u^2 at u=x^2+1#

03A negative inner rate can reverse the final sign#

04A prediction followed by a loss is a composition#

Evaluate the outer rate at the inner output, then multiply by the inner rate.

Sources & further reading

Reference this concept