Skip to content
Glacius
OptimizationConcept reference

First-order optimality

A differentiable interior local minimum must have zero gradient.

2 min read1 figureSources & further reading
On this page 8 sections
  1. Overview
  2. Understand the idea
  3. A closer look
  4. A zero gradient can occur at a minimum, maximum or saddle point
  5. An optimizer often stops near a zero gradient
  6. Key takeaway
  7. Sources & further reading
  8. Concept connections

01Understand the idea#

At a smooth unconstrained local minimum inside the domain, no sufficiently small movement can improve the objective to first order. The gradient must therefore be zero. This gives candidate points to investigate, rather than an automatic certificate of a minimum.

At a differentiable interior local minimum, the gradient must be zero. Each partial derivative must vanish.

This condition supplies candidates. You still need evidence to classify a stationary point.

Set both components of (2x−10,4y+4) to zero. Solving gives the stationary candidate (5,−1).

Gradient components are 2x−10 and 4y+4. The first is zero on x=5; the second is zero on y=−1. Their intersection is (5,−1), the stationary candidate. The diagram visualizes this gradient test, not a general minimum certificate.Gradient components are 2x−10 and 4y+4. The first is zero on x=5; the second is zero on y=−1. Their intersection is (5,−1), the stationary candidate. The diagram visualizes this gradient test, not a general minimum certificate.
Figure 1Gradient components are 2x−10 and 4y+4. The first is zero on x=5; the second is zero on y=−1. Their intersection is (5,−1), the stationary candidate. The diagram visualizes this gradient test, not a general minimum certificate.
Link to this figure ↗Download SVGDownload PNG

For f(x)=(x−2)2f(x)=(x-2)^2, the derivative 2(x−2)2(x-2) vanishes at 2 and the square is minimized there. For g(x)=−x2g(x)=-x^2, the derivative also vanishes at 0, but moving either way lowers the function. The same zero-gradient condition can describe a minimum or a maximum.

Check your reasoning

∇f(x,y)=(4x−12,6y+12)\nabla f(x,y)=(4x-12, 6y+12). Find the stationary candidate (x,y).

  1. A(3,2)(3, 2)
  2. B(3,−2)(3, -2)
  3. C(−3,−2)(-3, -2)
Show answer and explanation
(3,−2)(3, -2)

Set both components to zero: (x,y)=(3, -2).

02A closer look#

If a minimum location is specified but a coefficient is unknown, substitute that location into the derivative and set it to zero.

Keep the differentiable-interior assumptions explicit when using this necessary condition.

Check your reasoning

f(x)=5x2+bxf(x)=5x^2+bx on the real line. If x=−1x=-1 is an interior minimum, what must b equal?

Show answer and explanation
10

The derivative is 10x+b. At x=-1, zero requires b=10.

03A zero gradient can occur at a minimum, maximum or saddle point#

A zero gradient can occur at a minimum, maximum or saddle point. At a saddle, nearby points have both higher and lower values.

Boundary minima and nondifferentiable minima can fail the zero-gradient test because its assumptions do not apply.

Check your reasoning

f(x,y)=x2−y2f(x,y)=x^2-y^2 has zero gradient at (0,0). A report calls it a minimum. Repair it.

  1. AIt is a minimum.
  2. BIts gradient is nonzero.
  3. CIt is a saddle point.
Show answer and explanation
It is a saddle point.

Along y: lower; x: higher.

04An optimizer often stops near a zero gradient#

An optimizer often stops near a zero gradient. To interpret that point, consider the objective and constraints. A boundary minimum can have a nonzero ordinary derivative because some improvement directions lie outside the allowed domain.

Key takeaway

Set every gradient component to zero, then check classification and the interior/differentiability assumptions.

  • Apply the zero-gradient necessary condition to a differentiable interior local optimum.

Sources & further reading

  1. [1]
    OpenStax Calculus Volume3 §4.7 ↗OpenStax Calculus Volume3 §4.7 · Article

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “First-order optimality.” Math behind ML. /learn/o-first-order