The cross-entropy derivative, one logit at a time

Suppose a classifier gives the correct class a probability of $0.3$. Its cross-entropy loss is $-\ln(0.3)\approx1.204$. Differentiate that loss with respect to the probability and you get $-1/0.3\approx-3.333$. Differentiate it with respect to the logit that produced the probability and you get $0.3-1=-0.7$.

Both derivatives are correct. They describe changes to different inputs. The familiar cross-entropy gradient $p-y$ is with respect to logits, after composing the loss with softmax. For one unweighted example, natural logarithms, and a fixed target distribution that sums to one,

$$\frac{\partial L}{\partial z_j}=p_j-y_j, \qquad p=\operatorname{softmax}(z).$$

We will calculate those derivatives for three classes, work through why every logit matters, and check the result by perturbing the inputs in Python.

A three-class prediction you can calculate by hand

A logit is a real-valued score before softmax converts the scores into probabilities. Choose

$$z=(\ln2,\ln5,\ln3), \qquad y=(0,0,1).$$

The target is class 3. Using logarithms for our starting scores makes the exponentials easy to check:

$$p_j=\frac{e^{z_j}}{\sum_k e^{z_k}}, \qquad p=\left(\frac2{10},\frac5{10},\frac3{10}\right).$$

The prediction favors class 2 even though the target is class 3. For fixed targets $y$, categorical cross-entropy is

$$L=-\sum_{i=1}^{3} y_i\ln p_i.$$

With our one-hot target, only the third term contributes: $L=-\ln p_3\approx1.203973$. The cross-entropy lesson explains this loss as the cost of assigning probability to outcomes; here we are interested in its sensitivity to the model's scores.

Subtract the target from the prediction one entry at a time:

The positive derivatives mean that increasing either wrong-class logit raises the loss locally. The negative derivative for class 3 means increasing that logit lowers it. Gradient descent subtracts the gradient, so a direct update to these scores would lower the first two and raise the third.

Derive the gradient by writing the loss in logits

The shortest route avoids a full softmax Jacobian. Take the logarithm of the probability:

$$\ln p_i=z_i-\ln\left(\sum_k e^{z_k}\right).$$

Substituting into the loss gives two terms:

$$L=-\sum_i y_i z_i +\left(\sum_i y_i\right)\ln\left(\sum_k e^{z_k}\right).$$

Because the targets sum to one, this becomes

$$L=\ln\left(\sum_k e^{z_k}\right)-\sum_i y_i z_i.$$

When we differentiate with respect to $z_j$, the log-sum-exp term contributes $e^{z_j}/\sum_k e^{z_k}=p_j$. The target-weighted sum contributes $y_j$. Therefore

$$\frac{\partial L}{\partial z_j} =\frac{e^{z_j}}{\sum_k e^{z_k}}-y_j =p_j-y_j.$$

This is the same calculation developed in Dive into Deep Learning's softmax regression chapter. The result uses the whole probability distribution: changing one logit changes the denominator shared by every class.

Why the derivative with respect to probability is different

Before passing through softmax, the loss has partial derivatives

$$\frac{\partial L}{\partial p_i}=-\frac{y_i}{p_i}.$$

This treats the positive coordinates of $p$ as separate inputs to the loss expression. The softmax operation then accounts for how those coordinates move together. For our example, the incoming gradient with respect to probabilities is $(0,0,-10/3)$.

The softmax Jacobian contains a derivative for each output probability with respect to each input logit:

$$\frac{\partial p_i}{\partial z_j} =p_i(\delta_{ij}-p_j).$$

Here $\delta_{ij}$ is one when $i=j$ and zero otherwise. Increasing a class's own logit increases its probability at rate $p_i(1-p_i)$; increasing another class's logit decreases it at rate $-p_i p_j$.

Only $p_3$ contributes directly to our one-hot loss. Its derivatives with respect to all three logits are

$$\left(\frac{\partial p_3}{\partial z_1}, \frac{\partial p_3}{\partial z_2}, \frac{\partial p_3}{\partial z_3}\right) =(-0.06,-0.15,0.21).$$

Multiplying by $\partial L/\partial p_3=-10/3$ yields $(0.2,0.5,-0.7)$ again. The wrong classes have nonzero logit gradients because both affect $p_3$ through softmax's denominator.

For a general normalized target, the multivariable chain rule sums all those paths:

$$\begin{aligned} \frac{\partial L}{\partial z_j} &=\sum_i\left(-\frac{y_i}{p_i}\right)p_i(\delta_{ij}-p_j)\\ &=-y_j+p_j\sum_i y_i\\ &=p_j-y_j. \end{aligned}$$

Multiplying $p-y$ by the softmax derivative a second time would differentiate through softmax twice. Once you have $p-y$, you already have the gradient at the logits.

Read the derivative as a local change in loss

Hold $z_1=\ln2$ and $z_2=\ln5$ fixed, and vary only the correct-class logit $z_3=t$. The loss becomes a function of one number:

$$L(t)=\ln(7+e^t)-t.$$

At $t=\ln3$, its slope is $3/10-1=-0.7$. Increasing this logit by a small amount $h$ changes the loss by approximately $-0.7h$. For example, $h=0.01$ predicts a change of $-0.007$; the exact change is about $-0.006989$.

Figure link ↗SVG ↓PNG ↓Data ↓

The tangent describes the nearby curve. Farther away, the probability changes and so does the derivative. The other two logits also have their own positive slopes at the starting point; this plot isolates the third coordinate rather than showing a full training trajectory.

As another check, the three logit derivatives sum to zero. Adding the same constant to every logit leaves softmax unchanged, so it must leave the loss unchanged as well.

Verify the calculation with stable Python

This example uses only Python's standard library. Subtracting the largest logit keeps the exponentials at or below one. Computing the loss from the shifted logits avoids taking the logarithm of a probability that may have rounded to zero.

import math


def cross_entropy(logits, target):
    if not logits or len(logits) != len(target):
        raise ValueError("Use equal, nonempty vectors")
    if not all(math.isfinite(v) for v in [*logits, *target]):
        raise ValueError("Inputs must be finite")
    if any(v < 0 for v in target) or not math.isclose(
        math.fsum(target), 1.0, rel_tol=0, abs_tol=1e-12
    ):
        raise ValueError("Target must be a probability distribution")

    largest = max(logits)
    shifted = [v - largest for v in logits]
    exponentials = [math.exp(v) for v in shifted]
    total = math.fsum(exponentials)
    probabilities = [v / total for v in exponentials]
    loss = math.log(total) - math.fsum(
        y * z for y, z in zip(target, shifted)
    )
    gradient = [p - y for p, y in zip(probabilities, target)]
    return loss, probabilities, gradient


z = [math.log(2), math.log(5), math.log(3)]
y = [0.0, 0.0, 1.0]
loss, p, gradient = cross_entropy(z, y)

# Central differences perturb one logit while fixing the others.
h = 1e-5
numeric = []
for j in range(len(z)):
    plus, minus = z.copy(), z.copy()
    plus[j] += h
    minus[j] -= h
    numeric.append(
        (cross_entropy(plus, y)[0] - cross_entropy(minus, y)[0])
        / (2 * h)
    )

assert max(abs(a - b) for a, b in zip(gradient, numeric)) < 1e-8
print("probabilities:", [round(v, 6) for v in p])
print("analytic:", [round(v, 6) for v in gradient])
print("numeric: ", [round(v, 6) for v in numeric])
print(f"loss: {loss:.6f}")

# A direct step in logit space, not a network parameter update.
new_z = [v - 0.5 * g for v, g in zip(z, gradient)]
print(f"loss after step: {cross_entropy(new_z, y)[0]:.6f}")

Both gradient lines print [0.2, 0.5, -0.7]. The loss changes from 1.203973 to 0.850053 after the direct logit update. The finite-difference check estimates each derivative using only evaluations of the loss, independently of the $p-y$ expression.

In a neural network, the logits depend on weights and earlier activations. Backpropagation carries this gradient through those operations to the parameters before the optimizer takes a step. Our backpropagation and gradient descent example follows that process through an affine layer. Its squared-error loss has a different starting derivative; the same chain-rule machinery carries it backward.

Soft labels, class weights, and batch means

Soft labels still give p minus y

Nothing in the derivation required a single entry of $y$ to equal one. It required $\sum_i y_i=1$, with the targets held fixed while differentiating the logits. If our target becomes $(0.1,0.2,0.7)$, the gradient at the same prediction is $(0.1,0.3,-0.4)$.

For label smoothing, substitute the actual smoothed target into $p-y$. Do not keep subtracting the original one-hot vector. TensorFlow's softmax cross-entropy documentation likewise requires each label vector to form a valid probability distribution.

Class weights change the expression

If the per-example loss is instead

$$L_w=-\sum_i w_i y_i\ln p_i,$$

the same differentiation gives

$$\frac{\partial L_w}{\partial z_j} =p_j\sum_i w_i y_i-w_jy_j.$$

For a one-hot target in class $c$, this simplifies to $w_c(p_j-y_j)$: the target class's weight scales the entire gradient. For soft targets, multiplying each entry of $p-y$ by its own class weight generally gives the wrong answer.

For instance, use $w=(1,2,3)$ and soft target $y=(0.1,0.2,0.7)$. Then $\sum_i w_i y_i=2.6$, and the weighted gradient at $p=(0.2,0.5,0.3)$ is $(0.42,0.90,-1.32)$. It still sums to zero.

A mean introduces a denominator

For an unweighted batch of $B$ examples with $L_{\text{mean}}=B^{-1}\sum_n L_n$, the derivative for row $n$ is $(p_n-y_n)/B$. A summed loss has no $1/B$ factor. This assumes each row contributes one classification loss and none are ignored.

Framework reductions deserve a check when using weights. PyTorch's weighted mean with class-index targets divides by the sum of the non-ignored target weights. With probability targets, its mean divides by the number of loss elements. These denominators can produce different gradient scales even when the unreduced losses match. See the explicit formulas in PyTorch's CrossEntropyLoss reference.

Match the formula to the loss you actually call

PyTorch's CrossEntropyLoss and TensorFlow's softmax_cross_entropy_with_logits accept logits and perform the required normalization internally. Passing softmax probabilities as their logit input changes the function being optimized. TensorFlow documents this input requirement explicitly.

For binary classification with one sigmoid output, the analogous result is also $p-y$ with respect to its logit. The probability derivative differs: for $L=-y\ln p-(1-y)\ln(1-p)$, it is $(p-y)/(p(1-p))$. The sigmoid derivative supplies the cancelling factor $p(1-p)$. Independent sigmoid outputs for multilabel classification therefore need their own loss setup, rather than one softmax across all labels.

All calculations here use natural logarithms. A base-two loss and its derivatives are larger by $1/\ln2$. When checking a mismatch in your own implementation, first identify the differentiated variable, the target distribution, and the reduction. The short expression $p-y$ is useful precisely because those choices are explicit.