Machine learning
The cross-entropy derivative, one logit at a time
Derive softmax cross-entropy with respect to logits, check p − y with a three-class example, and see what changes with soft labels, weights, and batch means.
Read articleThe Glacius Blog
Machine learning
Worked explanations of the mathematics behind machine learning, with diagrams and calculations you can follow.
Follow via RSSMore to explore
6 articlesMachine learning
Gradient clipping by norm vs value: what does the limit mean?
Clip the same gradient two ways, see why their lengths differ, and check global clipping, PyTorch code, and the limits of what clipping can fix.
Read articleMachine learning
Normalization vs standardization: what happens to an outlier?
Compare min–max scaling and z-scores on one small column. See why skew survives, how outliers change both methods, and why test values can exceed 1.
Read articleMachine learning
Ridge vs lasso: why one coefficient becomes zero
Follow the same two coefficients through ridge and lasso, derive the zero threshold, and see why correlated features complicate feature selection.
Read articleMachine learning
What backpropagation does before gradient descent takes a step
Work through backpropagation and a gradient descent update in matrix notation, with predictions, numerical gradients, dimension checks, and NumPy code.
Read articleMachine learning
Why gradient descent can oscillate
Why gradient descent oscillates: a visual, worked example showing convergence, overshooting, and divergence as the learning rate changes.
Read article