Skip to content
Glacius

Builds on functions

Calculus

Explore change, gradients, and the chain rule that makes learning algorithms possible.

∇f
Explore gradients

Inside this subject

Read at your own pace
01Limits and continuity4 lessons
  1. 01
    Average rate of change

    Compute an average rate of change over an interval.

  2. 02
    Approaching a limit

    Infer a finite limit from nearby values while distinguishing the endpoint value.

  3. 03
    One-sided limits

    Determine a one-sided limit from a piecewise graph.

  4. 04
    Limit algebra

    Evaluate a finite limit using valid algebraic limit laws.

02The derivative5 lessons
  1. 01
    Difference quotients

    Form the difference quotient for a scalar function.

  2. 02
    Derivative definition

    Compute a simple derivative from a difference-quotient limit.

  3. 03
    Corners and cusps

    Diagnose failure of differentiability from unequal one-sided slopes.

  4. 04
    Tangent lines

    Construct a tangent-line equation from a point and derivative.

  5. 05
    Local linearization

    Approximate a nearby function value using a local linearization.

03Algebraic derivative rules6 lessons
  1. 01
    Constant rule

    Differentiate a constant scalar function.

  2. 02
    Power rule

    Differentiate a scalar power function on its real domain.

  3. 03
    Sum rule

    Differentiate a sum of differentiable scalar functions.

  4. 04
    Constant multiple rule

    Differentiate a constant multiple of a differentiable function.

  5. 05
    Product rule

    Differentiate a product of two scalar functions.

  6. 06
    Scalar chain rule

    Differentiate a composition of two scalar functions.

04Exponential and trigonometric change2 lessons
  1. 01
    Exponential derivatives

    Differentiate an exponential function with a constant base.

  2. 02
    Logarithm derivatives

    Differentiate a natural logarithm on its real domain.

05Shape and approximation1 lessons
  1. 01
    Second derivatives

    Compute the second derivative of a scalar function.

06Functions of several variables2 lessons
  1. 01
    Multivariable functions

    Evaluate a scalar function at a vector input.

  2. 02
    Level sets

    Identify inputs sharing the same scalar output.

07Partial derivatives and gradients3 lessons
  1. 01
    Partial derivatives

    Compute a partial derivative with other coordinates held fixed.

  2. 02
    Gradients

    Assemble a gradient from partial derivatives in coordinate order.

  3. 03
    Directional derivatives

    Compute a directional rate using a unit direction.

08Vector derivatives and curvature3 lessons
  1. 01
    Total differentials

    Estimate a scalar change from a small vector perturbation.

  2. 02
    Jacobians

    Construct the Jacobian of a vector-valued map.

  3. 03
    Multivariable chain rule

    Compute the derivative of a composed vector map using Jacobians.

09Automatic differentiation8 lessons
  1. 01
    Computational graphs

    Represent a scalar expression as a directed graph of elementary operations.

  2. 02
    Forward evaluation

    Evaluate a small computational graph in dependency order.

  3. 03
    Local derivatives

    Compute one graph operation's local derivative with respect to an input.

  4. 04
    Reverse-mode differentiation

    Propagate a scalar-output sensitivity backward through a chain.

  5. 05
    Shared-path gradients

    Accumulate sensitivities at an input used by multiple graph branches.

  6. 06
    Vector–Jacobian products

    Propagate a supplied output sensitivity through a vector operation.

  7. 07
    Finite differences

    Estimate a derivative with a centered finite difference.

  8. 08
    Gradient checks

    Diagnose a mismatch between analytic and numerical derivatives using a stated tolerance.

10Gradients of Vector Objectives3 lessons
  1. 01
    Linear-Form Gradients

    Compute the gradient of aᵀx with respect to x for a constant real vector a.

  2. 02
    Quadratic-Form Gradients

    Compute the gradient of xᵀAx as (A + Aᵀ)x for a constant real square matrix A.

  3. 03
    Least-Squares Gradients

    Compute the gradient of one-half the squared Euclidean residual norm ||Ax − b||² as Aᵀ(Ax − b) for fixed A and b.

11Matrix Gradients and Backpropagation5 lessons
  1. 01
    Matrix-Shaped Gradients

    Assemble the gradient of a scalar function of a real matrix from its entrywise partial derivatives.

  2. 02
    Squared Frobenius-Loss Gradients

    Compute ∇_Z [||Z − Y||_F²/(2m)] = (Z − Y)/m for fixed real Y and a fixed positive scalar m.

  3. 03
    Local Change from a Matrix Gradient

    Estimate the first-order change of a scalar objective under a small matrix perturbation using the Frobenius inner product of its gradient and that perturbation.

  4. 04
    Matrix Product: Left-Input Gradient

    Compute the left-input gradient GBᵀ for Z = AB from a supplied incoming scalar-loss gradient G.

  5. 05
    Matrix Product: Right-Input Gradient

    Compute the right-input gradient AᵀG for Z = AB from a supplied incoming scalar-loss gradient G.