Skip to content
Glacius
StatisticsConcept reference

Confidence interval meaning

Confidence describes how often an interval-producing procedure covers a fixed population parameter under its stated sampling model.

2 min read1 figureSources & further reading
On this page 8 sections
  1. Overview
  2. Understand the idea
  3. A closer look
  4. A closer look
  5. A closer look
  6. Key takeaway
  7. Sources & further reading
  8. Concept connections

01Understand the idea#

A confidence interval comes from a procedure designed to cover a fixed population parameter at a stated rate across repeated samples. The sample changes, so the interval changes. This describes the procedure’s reliability; no particular observed interval is guaranteed to contain the truth.

A confidence procedure produces an interval from random sample data. Across repetitions, the interval changes while the population parameter stays fixed. Coverage means that the interval contains that fixed parameter.

At fixed target 4, the three interval outputs have the listed probabilities. The last two contain 4, including the endpoint of [4,7][4,7].

At fixed parameter 4, three possible closed interval outputs are [1,3] with probability 1/4,[2,5] with probability 1/2, and[4,7] with probability 1/4. The first misses 4; the other two contain 4, including the endpoint 4. Their total probability is 3/4. All rows use the same horizontal numeric scale; the dashed parameter line remains fixed.At fixed parameter 4, three possible closed interval outputs are [1,3] with probability 1/4,[2,5] with probability 1/2, and[4,7] with probability 1/4. The first misses 4; the other two contain 4, including the endpoint 4. Their total probability is 3/4. All rows use the same horizontal numeric scale; the dashed parameter line remains fixed.
Figure 1At fixed parameter 4, three possible closed interval outputs are [1,3] with probability 1/4,[2,5] with probability 1/2, and[4,7] with probability 1/4. The first misses 4; the other two contain 4, including the endpoint 4. Their total probability is 3/4. All rows use the same horizontal numeric scale; the dashed parameter line remains fixed.
Link to this figure ↗Download SVGDownload PNG

Fix a target at 4. An interval [1,3][1,3] misses; [2,5][2,5] covers; and closed interval [4,7][4,7] also covers because an endpoint counts. If those three outputs have probabilities 1/4,1/2,1/41/4,1/2,1/4, add only the covering probabilities to get 3/43/4. The target stays 4 in every repetition.

At that target, coverage is 1/2+1/4=3/41/2+1/4=3/4. This calculation describes the supplied model at 44. A valid confidence guarantee must hold over the parameter values for which the procedure is claimed.

Check your reasoning

Fixed θ=3. All interval outputs and masses: [0,2]:0.4; [2,4]:0.2; [3,5]:0.4. Intervals are closed. Coverage here?

Show answer and explanation
0.6

Included masses: 0.2 + 0.4 = 0.6.

02A closer look#

In a simulation, a known target lets us mark each repeated interval as covering or missing. The target is shared across runs; it is not redrawn for each interval.

Check your reasoning

Simulation target θ=4. Runs: A[1,3]; B[3,6]; C[4,7]. Covered runs and target?

  1. A{A,B}, fixed target.
  2. B{B,C}, fixed target.
  3. C{B,C}, changing target.
Show answer and explanation
{B,C}, fixed target.

B,C contain the same target 4.

03A closer look#

A 95% procedure describes repeated coverage, not a guaranteed count of 95 hits in every 100 repetitions. Once an interval is observed, it either contains the fixed target or does not; the target may remain unknown.

Check your reasoning

80% confidence interval: [4,9]. Fixed unknown target. “80% chance here.” Repair.

  1. A80% procedure coverage.
  2. B80% target chance.
  3. C80% of observations.
Show answer and explanation
80% procedure coverage.

Repeated coverage.

04A closer look#

For an estimated average model error, a confidence procedure describes repeated-sampling reliability under its assumptions. It is not the chance that an individual future error falls inside the interval. Prediction intervals answer that different question.

Key takeaway

The interval varies across repeated samples; the parameter stays fixed. Confidence describes procedure coverage, not a new probability distribution for an observed fixed target.

  • Interpret repeated coverage of a fixed parameter.

Sources & further reading

  1. [1]

Reference this concept

Link to this page, a section, or an individual figure.

Glacius. “Confidence interval meaning.” Math behind ML. /learn/s-confidence