In probability theory, a probability distribution is a mathematical function that provides the probabilities of occurrence of different possible outcomes in an experiment. These distributions are either discrete or continuous, depending on whether the outcomes are countable or measurable over a range of values.
This blog explores various probability distributions, their formulas, cumulative distribution functions (CDFs), probability mass functions (PMFs), and probability density functions (PDFs), with detailed examples for each distribution.
—
Discrete Probability Distributions
Bernoulli Distribution
The Bernoulli distribution represents a random experiment with exactly two outcomes, often called a “success” and a “failure”. It is characterized by a single parameter \( p \), which is the probability of success.
– PMF:
\[
P(X = x) = p^x (1 – p)^{1 – x}, \quad x \in \{0, 1\}
\]
where \( p \) is the probability of success and \( 1 – p \) is the probability of failure.
– Mean:
\[
E(X) = p
\]
– Variance:
\[
\text{Var}(X) = p(1 – p)
\]
Example: A coin toss is a Bernoulli distribution with \( p = 0.5 \), where the outcome is “Heads” (success) or “Tails” (failure).
—
Binomial Distribution
The Binomial distribution models the number of successes in a fixed number of independent Bernoulli trials. It is defined by two parameters: \( n \) (the number of trials) and \( p \) (the probability of success in each trial).
– PMF:
\[
P(X = x) = \binom{n}{x} p^x (1 – p)^{n – x}, \quad x = 0, 1, 2, \dots, n
\]
– Mean:
\[
E(X) = np
\]
– Variance:
\[
\text{Var}(X) = np(1 – p)
\]
Example: The probability of getting exactly 3 heads in 5 flips of a fair coin, where \( n = 5 \) and \( p = 0.5 \), is calculated as:
\[
P(X = 3) = \binom{5}{3} (0.5)^3 (0.5)^{5 – 3} = 10 \times (0.5)^5 = 0.3125
\]
Poisson Distribution
The Poisson distribution models the number of events occurring within a fixed interval of time or space, where the events happen with a known constant mean rate \( \lambda \) and independently of the time since the last event.
– PMF:
\[
P(X = x) = \frac{\lambda^x e^{-\lambda}}{x!}, \quad x = 0, 1, 2, \dots
\]
– Mean:
\[
E(X) = \lambda
\]
– Variance:
\[
\text{Var}(X) = \lambda
\]
Example: If the average number of cars passing through a toll booth per minute is 3 (\( \lambda = 3 \)), the probability of exactly 5 cars passing through in a given minute is:
\[
P(X = 5) = \frac{3^5 e^{-3}}{5!} \approx 0.1008
\]
—
Geometric Distribution
The Geometric distribution models the number of trials needed to get the first success in a sequence of independent Bernoulli trials. It is characterized by a parameter \( p \), the probability of success on each trial.
– PMF:
\[
P(X = x) = (1 – p)^{x-1} p, \quad x = 1, 2, 3, \dots
\]
– Mean:
\[
E(X) = \frac{1}{p}
\]
– Variance:
\[
\text{Var}(X) = \frac{1 – p}{p^2}
\]
Example: The number of tosses needed to get the first “Heads” in a fair coin toss is geometrically distributed with \( p = 0.5 \).
—
Negative Binomial Distribution
The Negative Binomial distribution generalizes the geometric distribution. It models the number of trials needed to achieve \( r \) successes in independent Bernoulli trials.
– PMF:
\[
P(X = x) = \binom{x-1}{r-1} p^r (1 – p)^{x – r}, \quad x = r, r+1, r+2, \dots
\]
– Mean:
\[
E(X) = \frac{r}{p}
\]
– Variance:
\[
\text{Var}(X) = \frac{r(1 – p)}{p^2}
\]
Example: In a sequence of coin tosses, the number of tosses needed to get 3 heads (with \( p = 0.5 \)) follows a negative binomial distribution.
—
Continuous Probability Distributions
Uniform Distribution
The Uniform distribution is the simplest continuous distribution. It assumes that all outcomes in a given range are equally likely.
– PDF:
\[
f(x) = \frac{1}{b – a}, \quad a \leq x \leq b
\]
– Mean:
\[
E(X) = \frac{a + b}{2}
\]
– Variance:
\[
\text{Var}(X) = \frac{(b – a)^2}{12}
\]
Example: The time it takes for a bus to arrive, which is uniformly distributed between 0 and 10 minutes, has the following PDF:
\[
f(x) = \frac{1}{10}, \quad 0 \leq x \leq 10
\]
—
Normal Distribution
The Normal distribution (or Gaussian distribution) is the most widely used continuous probability distribution. It is characterized by its bell-shaped curve, symmetric around its mean \( \mu \) and defined by its standard deviation \( \sigma \).
– PDF:
\[
f(x) = \frac{1}{\sigma \sqrt{2\pi}} e^{-\frac{(x – \mu)^2}{2\sigma^2}}
\]
– Mean:
\[
E(X) = \mu
\]
– Variance:
\[
\text{Var}(X) = \sigma^2
\]
Example: If the heights of people in a population follow a normal distribution with a mean height of 170 cm and a standard deviation of 10 cm, the probability that a person has a height between 160 cm and 180 cm is found using the CDF of the normal distribution.
—
Exponential Distribution
The Exponential distribution is used to model the time between events in a Poisson process, where events occur continuously and independently at a constant rate \( \lambda \).
– PDF:
\[
f(x) = \lambda e^{-\lambda x}, \quad x \geq 0
\]
– Mean:
\[
E(X) = \frac{1}{\lambda}
\]
– Variance:
\[
\text{Var}(X) = \frac{1}{\lambda^2}
\]
Example: The time between successive calls arriving at a call center, modeled with \( \lambda = 1 \) call per minute, follows an exponential distribution.
—
Gamma Distribution
The Gamma distribution generalizes the exponential distribution. It models the time until the \( k \)-th event in a Poisson process.
– PDF:
\[
f(x) = \frac{x^{k-1} e^{-\lambda x}}{\Gamma(k)} \lambda^k, \quad x \geq 0
\]
where \( k \) is the shape parameter and \( \lambda \) is the rate parameter.
– Mean:
\[
E(X) = \frac{k}{\lambda}
\]
– Variance:
\[
\text{Var}(X) = \frac{k}{\lambda^2}
\]
Example: If the time between phone calls at a customer service center follows a Gamma distribution with shape parameter \( k = 3 \) and rate \( \lambda = 2 \), the probability of receiving a call after a certain amount of time is modeled by this distribution.
—
Beta Distribution
The Beta distribution is used to model random variables that are constrained to the interval [0, 1], such as probabilities.
– PDF:
\[f(x) = \frac{x^{\alpha-1} (1 – x)^{\beta-1}}{B(\alpha, \beta)}, \quad 0 \leq x \leq1\]
where \( \alpha \) and \( \beta \) are shape parameters and \( B(\alpha, \beta) \) is the Beta function.
– Mean:
\[
E(X) = \frac{\alpha}{\alpha + \beta}
\]
– Variance:
\[
\text{Var}(X) = \frac{\alpha \beta}{(\alpha + \beta)^2 (\alpha + \beta + 1)}
\]
Example: The proportion of time a worker spends on a particular task, which is constrained between 0 and 1, may follow a Beta distribution with specific parameters \( \alpha \) and \( \beta \).
—
Conclusion
Understanding the various probability distributions, their formulas, and their applications is crucial in fields like statistics, machine learning, and data science. From discrete distributions like Bernoulli and Poisson, to continuous distributions like Normal and Exponential, each distribution plays a key role in modeling different kinds of random processes. By understanding and applying the formulas for PMFs, PDFs, CDFs, means, and variances, one can effectively model uncertainty and make informed predictions in real-world scenarios.
