Logistic probabilities

01A logistic function turns an unrestricted scalar score into a number between 0 and 1#

A logistic function turns an unrestricted scalar score into a number between 0 and 1. In a binary prediction model, that number represents the forecast probability of the class labeled 1. The score may come from a weighted feature sum.

A score can be any real number. This curve maps it into a probability: score 0 lands at 0.5, with negative and positive scores on opposite sides.

Link to this figure ↗Download SVGDownload PNGCopy linkCopy embed

The supplied logistic link uses the negative score inside the exponential. Add 1 in the denominator, then take the reciprocal.

At score $z=0$, $e^{-z}=1$, so $1/(1+e^{-z})=1/2$. At score $z=\log 3$ using the natural logarithm, $e^{-z}=1/3$, so the probability is $1/(1+1/3)=3/4$. A positive score therefore gives a probability above one half.

02At z=0, the exponential is 1, so p=1/(1+1)=0.5#

At $z=0$, the exponential is 1, so $p=1/(1+1)=0.5$. At $z=2$, use $e^{-2}\approx0.1353$: $p\approx1/1.1353=0.881$. Reversing the score sign gives the opposite probability.

03For an event score z=1+2x at x=1, first compute z=3#

For an event score $z=1+2x$ at $x=1$, first compute $z=3$. Then $e^{-3}\approx0.0498$ gives $p\approx0.953$. Apply the link once to the complete score. Finite scores never give exactly 0 or 1.

04Turning probabilities into class labels requires a separate decision threshold#

Turning probabilities into class labels requires a separate decision threshold. A forecast 0.75 does not guarantee this particular example belongs to class 1. Calibration and evaluation assess how such probabilities behave across cases.

Further questions