Machine learning
The cross-entropy derivative, one logit at a time
Derive softmax cross-entropy with respect to logits, check p − y with a three-class example, and see what changes with soft labels, weights, and batch means.
Read articleThe Glacius Blog
Mathematics for
machine learning
Worked explanations of the mathematics behind machine learning, with diagrams and calculations you can follow.
Follow via RSSMore to explore
8 articlesMachine learning
Gradient clipping by norm vs value: what does the limit mean?
Clip the same gradient two ways, see why their lengths differ, and check global clipping, PyTorch code, and the limits of what clipping can fix.
Read articleMachine learning
Normalization vs standardization: what happens to an outlier?
Compare min–max scaling and z-scores on one small column. See why skew survives, how outliers change both methods, and why test values can exceed 1.
Read articleMachine learning
Ridge vs lasso: why one coefficient becomes zero
Follow the same two coefficients through ridge and lasso, derive the zero threshold, and see why correlated features complicate feature selection.
Read articleMachine learning
What backpropagation does before gradient descent takes a step
Work through backpropagation and a gradient descent update in matrix notation, with predictions, numerical gradients, dimension checks, and NumPy code.
Read articleMathematical intuition
Why cosine similarity and Euclidean distance can disagree
Compare two neighbors by hand, then normalize the vectors to see exactly when cosine similarity and Euclidean distance produce the same ranking.
Read articleMathematical intuition
How vector length changes dot product and cosine similarity
Dot product vs cosine similarity, explained visually. See how vector length changes a similarity score, with a worked example and the normalization formula.
Read articleMachine learning
Why gradient descent can oscillate
Why gradient descent oscillates: a visual, worked example showing convergence, overshooting, and divergence as the learning rate changes.
Read article