Builds on functions
Calculus
Explore change, gradients, and the chain rule that makes learning algorithms possible.
Inside this subject
Read at your own pace01Limits and continuity4 lessons
- 01Average rate of change
Compute an average rate of change over an interval.
- 02Approaching a limit
Infer a finite limit from nearby values while distinguishing the endpoint value.
- 03One-sided limits
Determine a one-sided limit from a piecewise graph.
- 04Limit algebra
Evaluate a finite limit using valid algebraic limit laws.
02The derivative5 lessons
- 01Difference quotients
Form the difference quotient for a scalar function.
- 02Derivative definition
Compute a simple derivative from a difference-quotient limit.
- 03Corners and cusps
Diagnose failure of differentiability from unequal one-sided slopes.
- 04Tangent lines
Construct a tangent-line equation from a point and derivative.
- 05Local linearization
Approximate a nearby function value using a local linearization.
03Algebraic derivative rules6 lessons
- 01Constant rule
Differentiate a constant scalar function.
- 02Power rule
Differentiate a scalar power function on its real domain.
- 03Sum rule
Differentiate a sum of differentiable scalar functions.
- 04Constant multiple rule
Differentiate a constant multiple of a differentiable function.
- 05Product rule
Differentiate a product of two scalar functions.
- 06Scalar chain rule
Differentiate a composition of two scalar functions.
04Exponential and trigonometric change2 lessons
05Shape and approximation1 lessons
06Functions of several variables2 lessons
07Partial derivatives and gradients3 lessons
08Vector derivatives and curvature3 lessons
09Automatic differentiation8 lessons
- 01Computational graphs
Represent a scalar expression as a directed graph of elementary operations.
- 02Forward evaluation
Evaluate a small computational graph in dependency order.
- 03Local derivatives
Compute one graph operation's local derivative with respect to an input.
- 04Reverse-mode differentiation
Propagate a scalar-output sensitivity backward through a chain.
- 05Shared-path gradients
Accumulate sensitivities at an input used by multiple graph branches.
- 06Vector–Jacobian products
Propagate a supplied output sensitivity through a vector operation.
- 07Finite differences
Estimate a derivative with a centered finite difference.
- 08Gradient checks
Diagnose a mismatch between analytic and numerical derivatives using a stated tolerance.
10Gradients of Vector Objectives3 lessons
- 01Linear-Form Gradients
Compute the gradient of aᵀx with respect to x for a constant real vector a.
- 02Quadratic-Form Gradients
Compute the gradient of xᵀAx as (A + Aᵀ)x for a constant real square matrix A.
- 03Least-Squares Gradients
Compute the gradient of one-half the squared Euclidean residual norm ||Ax − b||² as Aᵀ(Ax − b) for fixed A and b.
11Matrix Gradients and Backpropagation5 lessons
- 01Matrix-Shaped Gradients
Assemble the gradient of a scalar function of a real matrix from its entrywise partial derivatives.
- 02Squared Frobenius-Loss Gradients
Compute ∇_Z [||Z − Y||_F²/(2m)] = (Z − Y)/m for fixed real Y and a fixed positive scalar m.
- 03Local Change from a Matrix Gradient
Estimate the first-order change of a scalar objective under a small matrix perturbation using the Frobenius inner product of its gradient and that perturbation.
- 04Matrix Product: Left-Input Gradient
Compute the left-input gradient GBᵀ for Z = AB from a supplied incoming scalar-loss gradient G.
- 05Matrix Product: Right-Input Gradient
Compute the right-input gradient AᵀG for Z = AB from a supplied incoming scalar-loss gradient G.