The lesson library
A clear path through
the mathematics.
Choose a subject, follow its ideas in order, or find the one concept you need.
238 lessons across 6 subjects
Start with the foundations.
Build from familiar numbers to the functions and notation used throughout the library.
Explore by subject
From foundations to applicationsA place to begin
Foundations
Get comfortable with the language of mathematics: numbers, algebra, functions, and graphs.
Builds on algebra
Linear algebra
Understand vectors, matrices, and the geometry behind data and its transformations.
Builds on functions
Calculus
Explore change, gradients, and the chain rule that makes learning algorithms possible.
Builds on algebra
Probability
Work with uncertainty, distributions, and the evidence that changes what you believe.
Builds on probability
Statistics
Move from observations to estimates, inference, regression, and model evaluation.
Builds on gradients
Optimization
Follow the steps that improve a model, from gradient descent to regularization.
Put the ideas into practice.
Read here at your own pace. Open Glacius for questions, feedback, and review.
No lessons found. Try a subject, like “vectors”, or a different term.
Foundations
- Signed numbers
Interpret a signed integer as a position relative to zero.
- Adding signed numbers
Add two signed integers.
- Subtracting signed numbers
Subtract one signed integer from another.
- Multiplying signed numbers
Multiply two signed integers.
- Dividing signed numbers
Divide signed integers when the quotient is an integer and the divisor is nonzero.
- Absolute values
Compute a real number’s distance from zero.
- Fractions
Interpret a fraction as a quotient with a nonzero denominator.
- Equivalent fractions
Generate an equivalent fraction by scaling its numerator and denominator by the same nonzero integer.
- Adding fractions
Add two fractions using a common denominator.
- Subtracting fractions
Subtract two fractions using a common denominator.
- Multiplying fractions
Multiply two fractions.
- Dividing fractions
Divide a fraction by a nonzero fraction.
- Powers
Evaluate a positive whole-number power.
- The zero exponent
Evaluate a nonzero base raised to the zero power.
- Negative exponents
Evaluate a nonzero base raised to a negative integer power.
- Multiplying powers
Rewrite a product of powers with the same nonzero base as one power.
- Dividing powers
Rewrite a quotient of powers with the same nonzero base as one power.
- A power raised to a power
Rewrite a power raised to an integer power as one power for a nonzero base.
- Square roots
Evaluate a real square root with its domain restriction.
- Rearranging equations
Isolate one scalar variable in a linear equation.
- Distributing products
Expand a scalar product over a sum.
- Factoring
Factor a common scalar factor from a sum.
- Inequalities
Solve a one-step scalar inequality.
- Coordinates
Locate a point from an ordered coordinate pair.
- Function notation
Evaluate a scalar function at a supplied input.
- Function domains
Determine which real inputs a scalar expression allows.
- Reading graphs
Read the output of a scalar function from its graph.
- Straight-line slope
Compute the slope between two distinct points.
- Affine functions
Evaluate a scalar affine rule from its slope and intercept.
- Function composition
Evaluate a composition in the correct order.
- Inverse functions
Recover an input using an invertible scalar function.
- Exponential functions
Predict multiplicative change from an exponential rule.
- Logarithms
Evaluate a logarithm as an inverse exponent.
- Logarithms of products
Rewrite a positive product using logarithms.
- Set membership
Decide whether an object belongs to a specified finite set.
- Set union
Form the union of two finite sets.
- Set intersection
Form the intersection of two finite sets.
- Summation notation
Expand and evaluate a finite indexed sum.
- Product notation
Expand and evaluate a finite indexed product.
- Quantifiers
Distinguish a universal claim from an existence claim.
- Counterexamples
Refute a universal claim with a valid counterexample.
Linear algebra
- Vector coordinates
Interpret the entries of a vector in a named coordinate system.
- Vector addition
Add two vectors of the same dimension.
- Scalar multiplication
Scale each coordinate of a vector by a scalar.
- Vector differences
Compute the displacement between two coordinate vectors.
- Linear combinations
Construct a weighted sum of supplied vectors.
- Dot products
Compute the dot product of two equal-length vectors.
- Euclidean length
Compute the Euclidean length of a vector.
- Unit vectors
Normalize a nonzero vector to unit length.
- Euclidean distance
Compute the Euclidean distance between two vectors.
- Orthogonal vectors
Decide whether two vectors are orthogonal using their dot product.
- Cosine similarity
Compute directional similarity for two nonzero vectors.
- Embedding similarity
Interpret a supplied embedding ranking within its stated representation.
- L1 norm
Compute the L1 norm of a vector.
- Matrix shape
Identify the row and column dimensions of a matrix.
- Matrix entries
Retrieve a matrix entry using its row and column indices.
- Matrix addition
Add matrices of equal shape.
- Matrix scaling
Multiply each matrix entry by a scalar.
- Matrix-vector products
Compute a matrix-vector product from row dot products.
- Column combinations
Express a matrix-vector product as a combination of its columns.
- Matrix products
Compute the entries of a compatible matrix product.
- Matrix transpose
Transpose a rectangular matrix.
- Diagonal matrices
Interpret the coordinate scaling performed by a diagonal matrix.
- Outer products
Construct an outer product from two vectors.
- Systems as matrices
Encode a finite linear system as Ax=b.
- Row operations
Apply one solution-preserving elementary row operation.
- Gaussian elimination
Eliminate a variable using elementary row operations.
- Pivots
Identify pivot positions in row-echelon form.
- Back substitution
Solve a triangular system by back substitution.
- Reduced echelon form
Reduce an echelon matrix to reduced row-echelon form.
- Inconsistent systems
Detect a contradictory row in an augmented system.
- Free variables
Parameterize the free variables of a consistent reduced system.
- Unique solutions
Determine whether a consistent system has a unique solution.
- Span
Decide whether a vector lies in the span of supplied vectors.
- Subspaces
Check the defining closure conditions for a candidate real subspace.
- Linear independence
Test whether a supplied vector list is linearly independent.
- Bases
Verify a basis for a specified finite-dimensional subspace.
- Dimension
Determine subspace dimension from a basis.
- Column space
Find a basis for the column space of a matrix.
- Matrix rank
Determine matrix rank from its pivots.
- Linearity
Check additivity and homogeneity for a candidate map.
- Composing maps
Represent a composition of linear maps as an ordered matrix product.
- Affine maps
Separate the linear part from the translation in an affine map.
- Projection onto a line
Compute the orthogonal projection onto a nonzero vector's span.
- Orthonormal bases
Verify that a supplied basis is orthonormal.
- Projection onto a subspace
Project a vector using a supplied orthonormal basis.
- Least-squares geometry
Identify the closest attainable output in an inconsistent linear system.
- QR factorization
Use supplied reduced QR factors to solve a full-column-rank least-squares problem.
- Normal equations
Form the normal equations for a linear least-squares problem.
- Singular value decomposition
Interpret the three transformations in a supplied SVD.
- Singular values
Infer rank from the nonzero singular values of a matrix.
- Truncated SVD
Construct a rank-k approximation from supplied singular components.
- Frobenius norm
Compute a matrix's Frobenius norm.
- Approximation error
Compute truncated-SVD Frobenius error from discarded singular values.
- Condition numbers
Interpret the ratio of largest to smallest nonzero singular values for a full-rank square system.
Calculus
- Average rate of change
Compute an average rate of change over an interval.
- Approaching a limit
Infer a finite limit from nearby values while distinguishing the endpoint value.
- One-sided limits
Determine a one-sided limit from a piecewise graph.
- Limit algebra
Evaluate a finite limit using valid algebraic limit laws.
- Difference quotients
Form the difference quotient for a scalar function.
- Derivative definition
Compute a simple derivative from a difference-quotient limit.
- Corners and cusps
Diagnose failure of differentiability from unequal one-sided slopes.
- Constant rule
Differentiate a constant scalar function.
- Power rule
Differentiate a scalar power function on its real domain.
- Sum rule
Differentiate a sum of differentiable scalar functions.
- Constant multiple rule
Differentiate a constant multiple of a differentiable function.
- Product rule
Differentiate a product of two scalar functions.
- Scalar chain rule
Differentiate a composition of two scalar functions.
- Second derivatives
Compute the second derivative of a scalar function.
- Multivariable functions
Evaluate a scalar function at a vector input.
- Level sets
Identify inputs sharing the same scalar output.
- Partial derivatives
Compute a partial derivative with other coordinates held fixed.
- Gradients
Assemble a gradient from partial derivatives in coordinate order.
- Directional derivatives
Compute a directional rate using a unit direction.
- Jacobians
Construct the Jacobian of a vector-valued map.
- Multivariable chain rule
Compute the derivative of a composed vector map using Jacobians.
- Computational graphs
Represent a scalar expression as a directed graph of elementary operations.
- Forward evaluation
Evaluate a small computational graph in dependency order.
- Local derivatives
Compute one graph operation's local derivative with respect to an input.
- Reverse-mode differentiation
Propagate a scalar-output sensitivity backward through a chain.
- Shared-path gradients
Accumulate sensitivities at an input used by multiple graph branches.
- Vector–Jacobian products
Propagate a supplied output sensitivity through a vector operation.
- Finite differences
Estimate a derivative with a centered finite difference.
- Gradient checks
Diagnose a mismatch between analytic and numerical derivatives using a stated tolerance.
Probability
- Sample spaces
Specify outcomes fine enough to describe a finite experiment.
- Events
Represent an event as a subset of a sample space.
- Probability assignments
Check a finite probability assignment for nonnegativity and unit total.
- Event probabilities
Compute an event probability by summing disjoint outcome masses.
- Union probabilities
Compute a two-event union probability with overlap correction.
- Disjoint events
Determine whether two events are mutually exclusive.
- Conditional probability
Compute an event probability within a positive-probability conditioning event.
- Probability multiplication
Compute an intersection using a conditional probability.
- Partitions
Verify that a collection of events partitions the sample space.
- Total probability
Compute an event probability across an exhaustive disjoint case split.
- Bayes' rule
Reverse a conditional probability using a prior and total evidence probability.
- Base rates
Explain a posterior result using the prevalence of the hypothesized event.
- Event independence
Check independence using the product condition.
- Conditional independence
Check independence within a stated conditioning event.
- Random variables
Define a numerical random variable on a finite sample space.
- Probability mass functions
Construct a discrete random variable's probability mass function.
- Cumulative distributions
Compute a discrete cumulative distribution at a threshold.
- Reading probabilities from a CDF
Compute P(a < X ≤ b) by subtracting supplied CDF values at the endpoints.
- Expected values
Compute a finite random variable's expectation.
- Expected transformed values
Compute the expectation of a function of a discrete random variable.
- Linearity of expectation
Compute the expectation of a linear combination without assuming independence.
- Variance
Compute variance as expected squared deviation from the mean.
- Standard deviation
Express spread in the original variable's units.
- Variance under scaling
Compute variance after a scalar shift and scale.
- Reading density curves
Interpret a supplied area under a continuous density curve as an interval probability.
- Normal distributions
Identify a normal model's location and scale parameters.
- Standardization
Convert a normal threshold to standard-normal units.
- Normal probabilities
Compute a normal interval probability from a supplied standard-normal CDF.
- Joint mass functions
Construct a joint discrete distribution over paired values.
- Discrete marginals
Marginalize a joint mass table over one variable.
- Independent variables
Check factorization of a discrete joint distribution into its marginals.
- Chi-squared distributions
Identify a chi-squared variable as a sum of squared independent standard-normal variables.
- Student’s t distributions
Interpret a supplied standard-normal to scaled-chi-squared ratio as a t variable under independence.
- Joint expectations
Compute an expectation of a function using a finite joint table.
- Covariance
Compute covariance from a joint distribution.
- Variance of sums
Compute variance of a sum with a supplied covariance.
- Covariance matrices
Assemble the covariance matrix of a finite random vector.
- IID samples
Check the identical-distribution and independence assumptions in a sampling story.
- Sample mean variability
Compute the variance of an IID sample mean with finite variance.
- Markov's inequality
Bound a nonnegative variable's upper tail from its mean.
- Chebyshev's inequality
Bound a deviation probability using finite variance.
- Law of large numbers
Interpret convergence of an IID sample average under a stated finite-variance assumption.
- Central limit theorem
Form a normal approximation for a standardized IID sum with finite positive variance.
- CLT limitations
Diagnose a normal approximation undermined by the sampling assumptions or heavy tails.
- Monte Carlo estimation
Estimate an expectation with a supplied set of independent simulated draws.
- Surprisal
Compute the information associated with a positive-probability event.
- Entropy
Compute entropy of a finite probability distribution.
- Cross-entropy
Compute expected negative log probability under a supplied target distribution.
- KL divergence
Compute a finite-distribution KL divergence with valid support handling.
Statistics
- Data variables
Classify an observed variable by its numerical or categorical meaning.
- Populations and samples
Identify the target population and observed sample in a study.
- Sample means
Compute a sample arithmetic mean.
- Sample medians
Compute a sample median using the stated even-size convention.
- Sample quantiles
Locate a sample quantile under a supplied convention.
- Sample variance
Compute sample variance using an explicitly stated denominator.
- Estimators
Distinguish a data-dependent estimator from its realized estimate.
- Sampling distributions
Identify the distribution induced by repeatedly sampling an estimator.
- Discrete likelihoods
Evaluate candidate parameters using a fixed discrete sample's probability model.
- Continuous likelihoods
Compare candidate parameters using a fixed continuous sample density.
- Log-likelihood
Express an independent-sample likelihood as a sum of log terms.
- MAP estimation
Select a posterior mode from a supplied likelihood and prior.
- Grid posteriors
Normalize posterior weights over a finite parameter grid.
- Standard errors
Interpret the standard deviation of an estimator's sampling distribution.
- Confidence interval meaning
Interpret confidence through repeated coverage of a fixed parameter.
- Mean intervals
Construct a one-sample t interval under stated sampling conditions.
- Bootstrap resampling
Construct a bootstrap sampling approximation for an IID sample statistic.
- Bootstrap intervals
Read a percentile interval from supplied bootstrap replicates with stated limitations.
- Null hypotheses
Specify a null and alternative for a stated population claim.
- Test statistics
Compute a supplied test statistic measuring discrepancy from a null.
- P-values
Compute a tail probability under a specified null distribution.
- Effect sizes
Interpret the magnitude of an effect in meaningful units.
- Multiple comparisons
Apply a Bonferroni familywise-error bound to a stated test family.
- Linear regression models
Specify a conditional mean model separating predictors from residual noise.
- Design matrices
Encode supplied predictors and an intercept as a design matrix.
- Fitting regression
Obtain coefficients for a full-rank regression using a supplied QR least-squares procedure.
- Regression residuals
Compute residuals from fitted predictions and observations.
- Collinearity
Diagnose coefficient instability caused by dependent or nearly dependent predictors.
- Holdout evaluation
Separate model fitting from an independent evaluation set.
- Cross-validation
Aggregate validation error from a supplied fold scheme without test-set reuse.
- Data leakage
Identify preprocessing or selection that uses unavailable evaluation information.
- Logistic probabilities
Convert a linear score to a probability through a supplied logistic link.
- Logistic log loss
Compute binary negative log-likelihood for logistic predictions.
- Probability calibration
Assess whether predicted probabilities match observed frequencies in supplied bins.
- Centering data
Center each data feature using training-sample means.
- Principal components
Find principal directions from a supplied SVD of centered data.
- Component scores
Project centered observations into a chosen principal-component basis.
- Explained variance
Compute a retained-variance fraction from supplied singular values.
Optimization
- Objective functions
Express a scalar objective for specified decision variables.
- Feasible sets
Identify decisions satisfying all stated constraints.
- Minima and minimizers
Distinguish the best objective value from the decision attaining it.
- Local and global optima
Distinguish neighborhood optimality from optimality over the feasible set.
- Empirical objectives
Form an average per-example loss over a fixed dataset.
- Convex sets
Check whether a line segment between feasible points remains feasible.
- Convex functions
Check the convexity inequality for a supplied scalar example.
- First-order optimality
Apply the zero-gradient necessary condition to a differentiable interior local optimum.
- Descent directions
Check whether a direction has a negative first-order objective change.
- Gradient descent steps
Compute one gradient descent update.
- Learning rates
Diagnose overshooting from a step size on a one-dimensional quadratic.
- Stopping criteria
Assess a proposed stopping decision using gradient size and supplied tolerances.
- Momentum
Compute a momentum update using a stated velocity convention.
- Sample gradients
Compute a single-example gradient for an empirical objective.
- Minibatch gradients
Average a minibatch's per-example gradients.
- Gradient noise
Interpret batch-size effects under a stated independent-gradient variance model.
- Stochastic gradient descent
Compute a parameter update using a supplied sampled gradient.
- Adaptive scaling
Compute an update with supplied per-coordinate second-moment scaling.
- Adam updates
Compute an Adam parameter update from supplied bias-corrected moment estimates.
- Moment bias correction
Correct a zero-initialized exponential moving average using a supplied formula.
- Gradient clipping
Rescale a gradient to satisfy a supplied norm threshold.
- L2 penalties
Compute the gradient contribution of a stated quadratic penalty.
- Ridge solutions
Solve a small penalized least-squares system with a stated treatment of the intercept.
- L1 penalties
Compute an L1-penalized objective for supplied coefficients.
- Subgradients
Identify valid subgradients of absolute value, including at zero.
- Soft thresholding
Apply a supplied soft-thresholding operator to one scalar coordinate.
- Proximal gradient steps
Compute a supplied proximal-gradient step for an L1-regularized objective.