Skip to content
Glacius
CalculusConcept reference

Total differentials

A scalar function’s gradient has one partial derivative per input coordinate.

2 min read0 figuresSources & further reading
On this page 7 sections
  1. Overview
  2. Add the local effects of several changes
  3. Compute the change, then the new value
  4. Local cancellation
  5. Key takeaway
  6. Sources & further reading
  7. Concept connections

01Add the local effects of several changes#

A scalar function’s gradient has one partial derivative per input coordinate. Each partial derivative predicts that coordinate’s contribution to a small output change. Add the contributions:

Δf≈∇f(x)TΔx.\Delta f\approx\nabla f(x)^T\Delta x.

This extends scalar linearization: one slope times one displacement becomes a dot product of sensitivities and coordinate displacements. Both vectors must use the same coordinate order.

02Compute the change, then the new value#

Let f(x,y)=x2+3yf(x,y)=x^2+3y. At (2,1)(2,1), its value is 77 and gradient is (4,3)(4,3). For perturbation (0.1,−0.2)(0.1,-0.2), the first contribution is 0.40.4 and the second is −0.6-0.6.

0.4−0.6=−0.2.0.4-0.6=-0.2.

The estimated new value is 7−0.2=6.87-0.2=6.8. The exact value at (2.1,0.8)(2.1,0.8) is 6.816.81. The difference of 0.010.01 comes from the quadratic term in the first input change.

Check your reasoning

The gradient is (2,−3) and a small perturbation is (0.1,0.2). Estimate the scalar change.

  1. A−0.4
  2. B0.8
  3. C−0.6
Show answer and explanation
−0.4

The dot product is 0.2−0.6=−0.4.

03Local cancellation#

If the dot product is zero, the first-order estimate is zero. The exact change can still be nonzero. At (0,0)(0,0), f(x,y)=x2+y2f(x,y)=x^2+y^2 has zero gradient, but moving to (h,0)(h,0) increases it by h2h^2.

Check your reasoning

A draft multiplies the two partial effects 0.2 and −0.3. What is their total first-order change?

  1. A−0.06
  2. B−0.1
  3. C0.5
Show answer and explanation
−0.1

First-order coordinate contributions are added.

Use the gradient at the original point and report whether the requested quantity is the change or the new value. The formula is exact for affine functions and generally approximate for finite steps through nonlinear functions.

Key takeaway

Estimate a scalar change from a small vector perturbation.

  • Estimate a scalar change from a small vector perturbation.

Sources & further reading

  1. [1]
  2. [2]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Total differentials.” Math behind ML. /learn/c-total-differential