The ideas behind the distributions
Each distribution is best remembered by its story: the random experiment that produces it. Learn the story and the formula, the relationships and the typical uses follow from it. Every section links to the Explorer, where you can simulate it.
What is a probability distribution?
A random variable X is a number whose value depends on chance: the number of heads in 10 flips, the time until the next customer, tomorrow's temperature. Its distribution says which values X can take (the support) and how likely each one is.
- Discrete variables take separate, countable values (0, 1, 2, …). They are described by a probability mass function (PMF), P(X = k), and the probabilities add up to 1.
- Continuous variables take any value in an interval. Any single exact value has probability 0, so they are described by a probability density function (PDF) f(x). Probability is the area under the density: P(a ≤ X ≤ b) = ∫ₐᵇ f(x) dx. A density can be larger than 1; only its total area must be 1.
- Both kinds have a cumulative distribution function (CDF), F(x) = P(X ≤ x), which climbs from 0 to 1. The CDF answers questions like "what fraction falls below this value?", and its inverse gives quantiles such as medians and percentiles.
Describing a distribution
- Mean (expected value) E[X]: the long-run average, the balance point of the distribution.
- Variance Var(X) = E[(X − E[X])²] and standard deviation √Var(X): the typical distance from the mean.
- Shape: symmetric or skewed, one hump or several, light or heavy tails. Heavy tails mean extreme values are much more common than a normal distribution would suggest. Some distributions (Cauchy) have tails so heavy that the mean does not exist.
- Parameters usually play one of three roles: location shifts the distribution (μ), scale stretches it (σ, θ, λ), and shape changes its form (k, α, β, ν).
Why simulation works
The Explorer draws thousands of random samples and compares them with the exact formulas. Two theorems explain why that comparison works:
- Law of large numbers: as the number of samples n grows, sample averages converge to the true mean, and the histogram converges to the true density. The error typically shrinks like 1/√n, so 100× more samples makes the histogram about 10× more accurate.
- Central limit theorem: the average of n independent draws with finite variance is approximately Normal(μ, σ²/n), whatever the original distribution. Try the CLT demo on the Relationships page.
Under the hood every sample starts as a Uniform(0, 1) number. Other distributions are produced by inverse-transform sampling (Exponential, Weibull, Cauchy), transformations (Box–Muller for the Normal), rejection sampling (Marsaglia–Tsang for the Gamma) or by building them from simpler ones (Beta from two Gammas, t from a Normal and a Chi-squared).
Choosing a distribution
Start from what the quantity is, not from what the data happen to look like:
| You are modelling… | Reach for |
|---|---|
| A single yes/no outcome | Bernoulli |
| Number of successes in a fixed number of trials | Binomial |
| Number of trials until the first / r-th success | Geometric / Negative binomial |
| Count of events in a fixed window of time or space | Poisson (overdispersed: Negative binomial) |
| Waiting time until the next event, constant rate | Exponential |
| Waiting time until the k-th event; positive skewed amounts | Gamma |
| Lifetimes whose failure rate changes with age | Weibull |
| A probability, rate or proportion in [0, 1] | Beta |
| Sum or average of many small independent effects | Normal |
| Product of many positive factors; positive and right-skewed | Log-normal |
| A bounded value with no other information | Uniform |
| A test statistic: mean with estimated σ / sum of squares / ratio of variances | Student t / Chi-squared / F |
| Heavy-tailed data with frequent outliers | Student t with small ν, or Cauchy |
Further reading
- J. K. Blitzstein & J. Hwang, Introduction to Probability, 2nd ed. (CRC Press, 2019). The "story" approach used on this page; the book is free at probabilitybook.net.
- L. M. Leemis & J. T. McQueston, "Univariate Distribution Relationships", The American Statistician 62(1), 45–53 (2008). The chart that inspired the Relationships page.
- NIST/SEMATECH e-Handbook of Statistical Methods, §1.3.6 "Probability Distributions" — itl.nist.gov.
- G. Casella & R. L. Berger, Statistical Inference, 2nd ed. (Duxbury, 2002), chapters 3 and 5.
- W. H. Press et al., Numerical Recipes, 3rd ed., chapter 6. The special functions used to compute the exact CDFs.