Probability

For any two events, the probability that at least one occurs is the sum of their probabilities minus the probability that both occur: P(A or B) = P(A) + P(B) - P(A and B). The subtraction avoids counting the overlap twice.

Example. If 40% of users use Android, 30% use a tablet and 10% use both, P(Android or tablet) = 0.4 + 0.3 - 0.1 = 0.6.

Related: Union of events, Mutually exclusive events, Intersection of events

The chance that an event happens at least once in several independent tries, found by taking 1 minus the chance it never happens: 1 - (1 - p)^n. It grows quickly with the number of tries.

Example. If each of 5 independent servers fails on a given day with probability 0.02, chance at least one fails is 1 - 0.98^5, about 0.096.

Related: Complement rule, Independence, Binomial distribution

The overall proportion of a population that has some characteristic, before considering any specific evidence. Ignoring it leads to badly wrong conclusions, especially when the characteristic is rare. It is the baseline against which any test result should be judged.

Example. If only 1 in 1,000 transactions is fraud, the base rate is 0.1%, so even a good detector produces mostly false alarms.

Related: Base rate fallacy, Prior probability, Bayes' theorem

The mistake of judging how likely something is by looking only at specific evidence and neglecting how common it is overall. It makes people overrate the meaning of a positive test for a rare condition.

Example. A 95%-accurate fraud alert on a category where 0.1% of transactions are fraud will still be wrong most of the time it fires.

Related: Base rate, Bayes' theorem, False positive rate

A formula for reversing a conditional probability: P(A | B) = P(B | A) x P(A) / P(B). It updates a starting belief about A in light of evidence B, and underlies spam filters, medical test interpretation and Bayesian machine learning.

Example. A disease affects 1% of people; a test catches 90% of cases and gives 9% false alarms. P(disease | positive) = 0.9 x 0.01 / (0.9 x 0.01 + 0.09 x 0.99), about 0.09.

Related: Prior probability, Posterior probability, Likelihood

An interpretation in which probability measures a degree of belief about a statement, updated as new evidence arrives. It allows probabilities for one-off situations that cannot be repeated, such as whether a particular model is correct.

Example. You believe there is a 30% chance a new feature will lift sign-ups. After a pilot shows a strong uplift, you revise the belief upward to, say, 70%.

Related: Prior probability, Posterior probability, Bayes' theorem

The distribution of a single yes-or-no trial: the variable takes the value 1 with probability p and 0 with probability 1 - p. Its mean is p and its variance is p(1 - p). It models any binary outcome.

Example. A customer clicks an ad with probability 0.03: X = 1 for a click, 0 otherwise.

Related: Binomial distribution, Random variable, Cross-entropy loss

The distribution of the number of successes in a fixed number n of independent trials, each with the same success probability p: P(k) = C(n, k) p^k (1 - p)^(n - k). Its mean is np and variance np(1 - p).

Example. If each of 10 emails is opened with probability 0.2, the chance exactly 3 are opened is C(10,3) x 0.2^3 x 0.8^7, about 0.20.

Related: Bernoulli distribution, Poisson distribution, Combination

A well-known counting result showing that in a group of only 23 people, the chance that at least two share a birthday exceeds 50%. It illustrates how the number of pairs grows quickly and is the basis of hash collision estimates.

Example. With 23 people there are 253 pairs, which is why a shared birthday is more likely than intuition suggests.

Related: Complement rule, Combination, Hash collision

The distribution of a single trial with several possible outcomes, each having its own probability, with the probabilities summing to 1. It generalises the Bernoulli distribution and is the output of a softmax classifier.

Example. A model gives probabilities 0.7, 0.2 and 0.1 for the classes cat, dog and bird for one image.

Related: Multinomial distribution, Softmax function, Bernoulli distribution

A theorem stating that the average (or sum) of many independent observations with finite variance is approximately normally distributed, whatever the shape of the original distribution. The spread of that average shrinks as sigma / square root of n.

Example. Individual order values may be very skewed, yet the average of 100 orders is roughly bell-shaped, letting us build confidence intervals.

Related: Normal distribution, Standard error, Law of large numbers

A way to write a joint probability as a product of conditional ones: P(A1, A2, ..., An) = P(A1) x P(A2 | A1) x ... x P(An | A1, ..., An-1). Language models use it to score a sentence word by word.

Example. P(the, cat, sat) = P(the) x P(cat | the) x P(sat | the, cat).

Related: Multiplication rule, Conditional probability, Joint probability

A guarantee that for any distribution with a finite mean and variance, the probability that a value lies more than k standard deviations from the mean is at most 1 / k^2. It gives bounds without assuming a bell shape.

Example. At most 1/4 = 25% of values can lie more than 2 standard deviations from the mean, whatever the distribution.

Related: Standard deviation, Markov's inequality, Empirical rule

The approach that assumes every outcome in the sample space is equally likely, so the probability of an event is the count of favourable outcomes divided by the count of all outcomes. It works well for fair dice, coins and cards but not when outcomes are unequal.

Example. P(drawing a heart from a standard 52-card deck) = 13 / 52 = 0.25.

Related: Frequentist probability, Sample space, Event

A design that divides the population into natural groups, randomly picks some whole groups and observes everyone in them. It is cheaper for spread-out populations but usually less precise than a simple random sample of the same size.

Example. To study schools, a researcher randomly picks 20 schools and surveys every student in them.

Related: Stratified sampling, Systematic sampling, Random sampling

A way of choosing k items out of n where order does not matter; the count is C(n, k) = n! / (k! (n - k)!). It is used to count equally likely outcomes.

Example. Choosing 3 team members from 10 can be done in C(10, 3) = 120 ways.

Related: Permutation, Factorial, Binomial distribution

The probability that an event does not happen equals 1 minus the probability that it does. It is often the easiest route when counting the opposite case is simpler, for example 'at least one' problems.

Example. If a server is up 99.5% of the time, P(down) = 1 - 0.995 = 0.005.

Related: Event, Probability axioms, At-least-one probability

The distribution of one variable when another is fixed at a specific value, found by slicing the joint distribution and rescaling so it sums to 1. Predictive models effectively learn this distribution of outputs given inputs.

Example. Among only Android users, the distribution of purchased versus not purchased is a conditional distribution.

Related: Joint distribution, Conditional probability, Marginal distribution

The average value of a random variable given that another variable is known or fixed at some value, written E[X | Y = y]. The best prediction of X from Y in terms of squared error is exactly this, which is why regression estimates it.

Example. The expected marks of students who studied 6 hours a day is a conditional expectation given study hours.

Related: Expected value, Conditional distribution, Law of total expectation

The probability that one event happens given that another event is known to have happened, written P(A | B) = P(A and B) / P(B). It restricts attention to the cases where the condition holds and is the foundation of learning from evidence.

Example. If 5% of emails are spam and contain 'lottery', while 6% contain 'lottery', P(spam | lottery) = 0.05 / 0.06, about 0.83.

Related: Bayes' theorem, Joint probability, Independence

A third variable that influences both the supposed cause and the supposed effect, creating a misleading association between them. Ignoring it can make a relationship appear where none exists. Controlling for it, or randomising, is how analysts separate real effects from spurious ones.

Example. Hot weather raises both ice cream sales and swimming accidents, so sales and accidents correlate without one causing the other.

Related: Correlation vs causation, Simpson's paradox, Independence

A random variable that can take any value in an interval of real numbers. The probability of any exact value is zero, so probabilities are found for ranges by integrating a density.

Example. A person's height or the response time of an API in milliseconds is continuous.

Related: Discrete random variable, Probability density function, Random variable

A number between -1 and 1 that measures the strength and direction of a straight-line relationship between two variables, found by dividing their covariance by the product of their standard deviations. It does not show that one causes the other.

Example. Ice cream sales and temperature might have correlation 0.9, meaning they rise together strongly in a near-linear way.

Related: Covariance, Correlation vs causation, Independence

The principle that two variables moving together does not prove one causes the other; the link may come from chance, a shared driver or reverse direction. Establishing cause typically needs a randomised experiment or careful causal reasoning.

Example. App usage and purchases are correlated, but heavy buyers may simply use the app more; only a randomised test shows whether a feature raised purchases.

Related: Confounding variable, Correlation coefficient, Selection bias

A measure of how two random variables move together: Cov(X, Y) = E[(X - E[X])(Y - E[Y])]. It is positive when they tend to be high or low together, negative when one is high as the other is low, and near zero when there is no linear relationship.

Example. Hours studied and exam marks typically have positive covariance; hours of TV and marks may have negative covariance.

Related: Correlation coefficient, Variance, Independence

A function giving the probability that a random variable is less than or equal to a given value: F(x) = P(X <= x). It rises from 0 to 1, works for both discrete and continuous variables, and is used to find percentiles and interval probabilities.

Example. For a die, F(4) = 4/6. Then P(2 < X <= 5) = F(5) - F(2) = 5/6 - 2/6 = 0.5.

Related: Probability density function, Quantile, Percentile

Events where the occurrence of one changes the probability of the other. Their joint probability cannot be found by simply multiplying the separate probabilities; the conditional probability must be used.

Example. Drawing cards without replacement: after one ace is taken, the chance that the next card is an ace falls from 4/52 to 3/51.

Related: Independence, Conditional probability, Multiplication rule

A random variable that takes a countable set of separate values, such as whole numbers. Its distribution is described by giving the probability of each value. Examples include counts of events, dice results and numbers of clicks, where values can be listed one by one.

Example. The number of customer complaints received in a day (0, 1, 2, ...) is discrete.

Related: Continuous random variable, Probability mass function, Random variable

An estimate of an event's probability calculated from observed data as the number of times it occurred divided by the number of trials. It approaches the true probability as the number of trials grows.

Example. A cricket batter scored a fifty in 12 of 80 innings, so the empirical probability of a fifty in an innings is 12 / 80 = 0.15.

Related: Frequentist probability, Law of large numbers, Relative frequency

A rule of thumb for bell-shaped data: roughly 68% of values lie within one standard deviation of the mean, 95% within two, and 99.7% within three. It gives a quick sense of what counts as typical or extreme.

Example. With mean order value Rs 500 and standard deviation Rs 100, about 95% of orders fall between Rs 300 and Rs 700.

Related: Normal distribution, Standard deviation, Z-score

A collection of outcomes of a random experiment that we care about, treated as a subset of the sample space. We say the event happens if the actual outcome belongs to that collection. Probability is assigned to events rather than to single outcomes.

Example. When rolling a die, the event 'even number' is {2, 4, 6} and has probability 3/6 = 0.5.

Related: Outcome, Sample space, Complement rule

In Bayes' theorem, the overall probability of the observed data averaged over all hypotheses, P(B) in P(A | B) = P(B | A) P(A) / P(B). It acts as a normalising constant that makes the posterior sum to 1.

Example. In the medical test example, P(positive) = 0.9 x 0.01 + 0.09 x 0.99, about 0.098, is the evidence term.

Related: Bayes' theorem, Law of total probability, Posterior probability

A set of events that together cover the entire sample space, so at least one of them must happen on every run of the experiment. When they are also mutually exclusive, their probabilities add up to exactly 1.

Example. For an email classifier, 'spam' and 'not spam' are exhaustive: every email falls into one of them.

Related: Partition, Mutually exclusive events, Sample space

The long-run average value of a random variable, computed by weighting each possible value by its probability: E[X] = sum of x x P(x). It is the centre of the distribution and drives decisions under uncertainty.

Example. A lottery ticket costing Rs 50 that pays Rs 1,000 with probability 0.02 has expected payout 0.02 x 1000 = Rs 20.

Related: Variance, Law of large numbers, Linearity of expectation

A continuous distribution for the waiting time until the next event in a process where events occur at a constant average rate lambda; density is lambda e^(-lambda x) and the mean is 1 / lambda. It is memoryless.

Example. If a server fails on average once every 200 hours, the time to next failure is exponential with mean 200 hours.

Related: Poisson distribution, Memoryless property, Geometric distribution

The product of all whole numbers from 1 up to n, written n!, with 0! defined as 1. It counts the number of ways to arrange n distinct items and appears in the formulas for permutations, combinations and the Poisson distribution.

Example. 5! = 5 x 4 x 3 x 2 x 1 = 120.

Related: Permutation, Combination, Poisson distribution

An interpretation in which the probability of an event is the long-run fraction of times it occurs when the same experiment is repeated many times. It ties probability to observable frequencies and underlies most classical statistical testing.

Example. If 1,000 loan applicants with a certain profile are tracked and 60 default, the estimated default probability is 60 / 1000 = 0.06.

Related: Bayesian probability, Law of large numbers, Empirical probability

The mistaken belief that after a run of one result in independent trials, the opposite result becomes more likely to balance things out. Independent trials have no memory, so past results do not change future probabilities.

Example. After five heads in a row, the chance of heads on a fair coin's next toss is still 0.5.

Related: Independence, Law of large numbers, Memoryless property

The distribution of the number of independent trials needed to get the first success when each trial succeeds with probability p: P(k) = (1 - p)^(k - 1) p. Its mean is 1 / p. It models waiting for a first event.

Example. If a sales call converts with probability 0.1, the average number of calls until the first sale is 1 / 0.1 = 10.

Related: Bernoulli distribution, Exponential distribution, Binomial distribution

A Monte Carlo technique that estimates an expectation under one distribution by sampling from another easier one and reweighting each sample by the ratio of the two densities. It helps with rare events and is used in off-policy reinforcement learning.

Example. To estimate the tiny chance of a very large loss, sample from a distribution that produces large losses more often and correct with weights.

Related: Monte Carlo simulation, Rejection sampling, Expected value

Two events are independent when knowing that one occurred does not change the probability of the other, so P(A and B) = P(A) x P(B). Independence is a common modelling assumption, and checking whether it really holds is an important part of data analysis.

Example. Two fair coin tosses are independent: P(both heads) = 0.5 x 0.5 = 0.25.

Related: Conditional independence, Multiplication rule, Correlation

The event that all of the given events occur together, written A and B. It contains only the outcomes shared by every event. Its probability appears in conditional probability and in the multiplication rule.

Example. With a die, the intersection of 'even' {2,4,6} and 'greater than 4' {5,6} is {6}, with probability 1/6.

Related: Union of events, Joint probability, Multiplication rule

A method for generating random values from any distribution: draw a uniform number u between 0 and 1 and return the quantile at u, that is, the inverse of the cumulative distribution function evaluated at u.

Example. For an exponential distribution with rate lambda, x = -ln(1 - u) / lambda turns uniform draws into exponential waiting times.

Related: Cumulative distribution function, Uniform distribution, Rejection sampling

A distribution that assigns probabilities to every combination of values of two or more random variables at once. From it one can obtain the marginal and conditional distributions, so it contains all information about how the variables relate.

Example. A table giving the probability of each pair (device type, purchased or not) is a joint distribution.

Related: Marginal distribution, Conditional distribution, Joint probability

The probability that two or more events both occur, written P(A and B). It describes how likely a particular combination of values is across several variables and is the starting point for marginal and conditional probabilities.

Example. If 12% of shoppers are both first-time visitors and use a coupon, the joint probability of 'first-time and coupon' is 0.12.

Related: Marginal probability, Conditional probability, Joint distribution

A measure of how one probability distribution P differs from a reference distribution Q, computed as the sum of P(x) x log(P(x) / Q(x)). It is zero only when they match, and it is not symmetric, so it is not a true distance.

Example. In training a variational model, KL divergence penalises the learned distribution for drifting from the prior.

Related: Entropy, Cross-entropy loss, Maximum likelihood estimation

A theorem stating that as more independent observations of the same random variable are collected, their sample average gets closer to its expected value. It justifies estimating probabilities and averages from data.

Example. Averaging 100,000 die rolls gives a number very near 3.5, while 5 rolls might give 2.6.

Related: Expected value, Central limit theorem, Sample mean

The overall expected value of a variable equals the average of its conditional expected values, weighted by the probabilities of the conditions: E[X] = sum of E[X | B] x P(B). It splits a hard average into easier cases.

Example. If mobile users spend Rs 300 on average (60% of users) and web users Rs 500 (40%), overall average is 0.6 x 300 + 0.4 x 500 = Rs 380.

Related: Law of total probability, Expected value, Conditional expectation

If events B1, B2, ... form a partition of the sample space, the probability of any event A is the sum over i of P(A | Bi) x P(Bi). It lets you compute an overall probability by averaging over separate cases.

Example. If 60% of orders come from mobile with 4% return rate and 40% from web with 2%, overall return probability = 0.6 x 0.04 + 0.4 x 0.02 = 0.032.

Related: Partition, Bayes' theorem, Marginal probability

How probable the observed data are for a particular value of a model's parameter, treated as a function of that parameter with the data held fixed. Unlike a probability distribution, it does not need to sum to 1 over parameter values.

Example. If a coin lands heads 7 times in 10 tosses, the likelihood at p = 0.7 is much higher than at p = 0.3.

Related: Maximum likelihood estimation, Posterior probability, Prior probability

The expected value of a sum of random variables equals the sum of their expected values, and E[aX + b] = a E[X] + b. It holds whether or not the variables are independent, which makes many calculations easy.

Example. If a delivery costs Rs 40 on average and a return costs Rs 25 on average, the expected total is Rs 65 regardless of any link between them.

Related: Expected value, Variance, Random variable

The distribution of one variable alone, obtained from a joint distribution by summing or integrating out all the other variables. It ignores what the others are doing. It answers questions about that variable when the others are of no interest.

Example. Adding across the 'purchased or not' columns of the device-and-purchase table leaves the distribution of device type.

Related: Joint distribution, Marginal probability, Conditional distribution

The probability of a single event or variable taken on its own, found by summing the joint probabilities over every value of the other variables. It is called marginal because in a table of counts it appears in the margins.

Example. If P(male, buys) = 0.2 and P(female, buys) = 0.3, then the marginal probability of buying is 0.2 + 0.3 = 0.5.

Related: Joint probability, Marginal distribution, Law of total probability

A sequence of random states in which the probability of the next state depends only on the current state and not on earlier history. It is described by a table of transition probabilities and models things like page browsing and text generation.

Example. If a user on a 'cart' page goes to 'checkout' with probability 0.3 regardless of how they arrived, that is the Markov property.

Related: Transition matrix, Stationary distribution, Markov property

A family of methods that draw samples from a complex distribution by building a Markov chain whose long-run distribution is the target. It is widely used in Bayesian inference when the posterior cannot be computed directly.

Example. The Metropolis-Hastings algorithm proposes a move, accepts or rejects it by a probability rule, and after many steps its visited states follow the target distribution.

Related: Stationary distribution, Posterior probability, Monte Carlo simulation

The assumption that, given the present state, the future is independent of the past. It simplifies modelling of sequences because only the current state needs to be tracked. It is the defining assumption of Markov chains and hidden Markov models.

Example. In weather modelling with this assumption, tomorrow's forecast depends on today's weather only, not on last week's.

Related: Markov chain, Conditional independence, Memoryless property

For a non-negative random variable, the probability of exceeding a value a is at most its expected value divided by a: P(X >= a) <= E[X] / a. It is a crude but widely applicable bound that needs only the mean.

Example. If average page load is 2 seconds, at most 2 / 10 = 20% of loads can take 10 seconds or more.

Related: Chebyshev's inequality, Expected value, Law of large numbers

A method of choosing model parameters as the values that make the observed data most probable, found by maximising the likelihood function or its logarithm. Many machine learning losses, such as cross-entropy, are derived from it.

Example. For 7 heads in 10 tosses, the value of p that maximises likelihood is 7 / 10 = 0.7.

Related: Likelihood, Maximum a posteriori estimation, Log-likelihood

The middle value of a distribution, so that half the probability lies at or below it and half at or above. Unlike the mean, it is not pulled around by extreme values, which makes it a robust measure of typical size.

Example. Salaries of 20k, 25k, 30k, 32k and 400k have a median of 30k, although their mean is far higher.

Related: Mean, Mode, Quantile

A distribution is memoryless when the chance of waiting a further amount of time does not depend on how long you have already waited. Among continuous distributions only the exponential has it, and among discrete ones only the geometric.

Example. If a bus arrival gap is exponential, having waited 10 minutes gives no clue that it is 'due' sooner.

Related: Exponential distribution, Geometric distribution, Markov chain

The most likely value of a random variable, that is, the value with the highest probability or the peak of the density. A distribution can have one mode, several modes or, for a flat distribution, none that stands out.

Example. In the sizes of shirts sold, if M sells most often, M is the mode.

Related: Mean, Median, Probability mass function

A technique that estimates a quantity by generating many random samples from a model and averaging the results. It is used when an exact calculation is too hard, and it approximates expectations and probabilities.

Example. To estimate pi, throw random points into a unit square: the fraction that fall inside a quarter circle, times 4, approaches 3.14 as points grow.

Related: Law of large numbers, Random number generator, Markov chain Monte Carlo

A famous puzzle in conditional probability: after a contestant picks one of three doors and the host opens another door that has no prize, switching wins with probability 2/3 while staying wins with 1/3. It shows how new information changes probabilities.

Example. Simulating 10,000 games shows switching wins about 6,667 times and staying about 3,333.

Related: Conditional probability, Bayes' theorem, Monte Carlo simulation

The probability that two events both occur equals the probability of the first times the conditional probability of the second given the first: P(A and B) = P(A) x P(B | A). For independent events it reduces to P(A) x P(B).

Example. Drawing two aces in a row without replacement: (4/52) x (3/51), about 0.0045.

Related: Conditional probability, Independence, Chain rule of probability

Events that cannot happen at the same time, because they share no outcomes. If one occurs, the other is ruled out. For such events the probability that either occurs is simply the sum of their probabilities.

Example. On a single die roll, 'roll a 2' and 'roll a 5' are mutually exclusive. P(2 or 5) = 1/6 + 1/6 = 1/3.

Related: Addition rule, Independence, Event

A symmetric, bell-shaped distribution fully described by its mean and standard deviation. About 68%, 95% and 99.7% of values lie within one, two and three standard deviations of the mean. It appears everywhere because sums of many small effects tend to look like it.

Example. Heights of adult men in a city might be normal with mean 170 cm and standard deviation 7 cm, so about 95% fall between 156 and 184 cm.

Related: Standard normal distribution, Central limit theorem, Z-score

One single, specific result of a random experiment. Outcomes are mutually exclusive: exactly one of them occurs each time the experiment is run. Events are built by grouping outcomes together.

Example. Rolling a die can produce the outcome 4. Tossing two coins can produce the outcome HT (heads then tails).

Related: Sample space, Event, Random experiment

A division of the sample space into events that are mutually exclusive and together exhaustive, so each outcome belongs to exactly one piece. Partitions are the basis of the law of total probability.

Example. Splitting customers by city tier into Tier-1, Tier-2 and Tier-3 is a partition: every customer is in exactly one group.

Related: Law of total probability, Exhaustive events, Mutually exclusive events

A value below which a given percentage of observations or of a distribution's probability lies. It is a quantile expressed on a 0 to 100 scale and is widely used for ranking and for describing tail latency.

Example. A score at the 95th percentile beats 95% of test takers; a p99 latency of 800 ms means 99% of requests are faster.

Related: Quantile, Median, Cumulative distribution function

An arrangement of items in a specific order. The number of ways to arrange k items chosen from n is n! / (n - k)!, and all n items can be ordered in n! ways. Order matters, unlike in combinations.

Example. Assigning gold, silver and bronze to 3 of 8 runners can be done in 8 x 7 x 6 = 336 ways.

Related: Combination, Factorial, Sample space

The distribution of how many events occur in a fixed interval of time or space when they happen independently at a constant average rate lambda: P(k) = e^(-lambda) lambda^k / k!. Both its mean and variance equal lambda.

Example. A helpline gets 4 calls a minute on average; the chance of exactly 2 calls in a minute is e^(-4) x 4^2 / 2, about 0.147.

Related: Exponential distribution, Binomial distribution, Poisson process

A model for events occurring randomly over time at a constant average rate, in which counts in separate intervals are independent and the gaps between events follow an exponential distribution. It is used for arrivals such as requests to a server.

Example. If requests arrive at 2 per second on average, the number in 10 seconds is Poisson with mean 20 and gaps average 0.5 seconds.

Related: Poisson distribution, Exponential distribution, Markov chain

The full set of individuals, items or events that a question is about, from which a sample may be drawn. Its numerical characteristics, such as its mean, are fixed but usually unknown.

Example. All registered voters in Karnataka form a population; a survey of 2,000 of them is a sample.

Related: Sample, Sampling, Sample mean

The updated probability of an event or parameter after taking the new evidence into account, computed by Bayes' theorem. It combines the prior with the likelihood and becomes the prior for the next round of updating.

Example. After a transaction is flagged by a fraud rule, the chance it is truly fraudulent rises from the 2% prior to a posterior of, say, 25%.

Related: Prior probability, Likelihood, Bayes' theorem

The probability assigned to an event or parameter before seeing the new evidence, reflecting earlier knowledge or assumption. In Bayes' theorem it is the starting belief that gets updated. It is combined with new data to produce an updated belief.

Example. Before testing, you assume 2% of transactions on a payments app are fraudulent; that 2% is the prior.

Related: Posterior probability, Likelihood, Bayes' theorem

A number between 0 and 1 that measures how likely an event is to happen, where 0 means impossible and 1 means certain. It gives a common language for uncertainty, and machine learning models use it to express how confident they are in a prediction.

Example. A fair coin lands heads with probability 0.5. If a UPI payment fails about 2 times in 100, the failure probability is 0.02.

Related: Event, Sample space, Random experiment

Three basic rules, laid down by Kolmogorov, from which all of probability follows: every event has probability of at least 0, the whole sample space has probability 1, and for events that cannot occur together the probability of any one of them happening is the sum of their individual probabilities.

Example. If P(rain) = 0.3 and P(snow) = 0 in Bengaluru, and they cannot co-occur, P(rain or snow) = 0.3 + 0 = 0.3. No probability may be negative or exceed 1.

Related: Sample space, Mutually exclusive events, Addition rule

A function describing how probability is spread over a continuous random variable. The probability of landing in an interval is the area under the curve over that interval, and the total area is 1. The height itself is not a probability.

Example. For a standard bell curve, the area under the density between -1 and 1 is about 0.68, so about 68% of values fall there.

Related: Probability mass function, Cumulative distribution function, Continuous random variable

A description of all the values a random variable can take together with how likely each is, either as a table, a formula or a curve. It summarises the whole pattern of uncertainty and is the object models are fitted to.

Example. For one die, each of the values 1 to 6 has probability 1/6. Daily sales might follow a bell-shaped distribution around 500 units.

Related: Probability mass function, Probability density function, Random variable

A function that gives the probability that a discrete random variable equals each specific value. Its values are never negative and add up to 1 over all possible values. It is the discrete counterpart of a density curve.

Example. For a fair die, the function gives 1/6 for each of 1 to 6, and these six values sum to 1.

Related: Probability density function, Discrete random variable, Cumulative distribution function

A value that splits a distribution so a stated fraction of the probability lies at or below it. The q-quantile is the value x where the cumulative distribution reaches q. It is found by inverting the cumulative distribution function.

Example. The 0.9 quantile of response time is the time below which 90% of requests fall; if it is 300 ms, then 90% of calls finish within 300 ms.

Related: Percentile, Median, Cumulative distribution function

A process whose result cannot be predicted with certainty in advance, but whose possible results are all known and which can in principle be repeated under the same conditions. Probability theory starts by describing such a process before assigning likelihoods to what might come out.

Example. Rolling a die, drawing a card, or checking whether the next customer who opens an app makes a purchase are all such processes.

Related: Outcome, Sample space, Trial

Selecting items from a population in a way that every item has a known, non-zero chance of being chosen, with the choice left to chance. It guards against personal bias and makes probability theory applicable to results.

Example. Assigning each of 10,000 users a random number and taking the 500 with the smallest numbers.

Related: Simple random sample, Sampling bias, Sample

A variable whose value is a numerical outcome of a random experiment. It maps each possible outcome to a number so that we can compute averages and spreads. It can be discrete or continuous.

Example. Let X be the number of heads in 3 coin tosses. X can be 0, 1, 2 or 3 depending on the outcome.

Related: Discrete random variable, Continuous random variable, Probability distribution

A technique for drawing samples from a target distribution by drawing from a simpler proposal distribution and accepting each draw with a probability proportional to the target density there. Rejected draws are discarded.

Example. To sample from an awkward curve, propose points under an easy envelope curve and keep only those that fall under the target curve.

Related: Inverse transform sampling, Importance sampling, Monte Carlo simulation

A subset of a population that is actually observed, used to learn about the whole. Its usefulness depends on how it was chosen; a well-chosen random sample tends to mirror the population.

Example. Surveying 500 randomly chosen customers out of 2 lakh to learn about satisfaction.

Related: Population, Sampling, Random sampling

The average of the values in a sample, found by adding them and dividing by how many there are. It is itself a random variable, varying from sample to sample, and is used to estimate the population mean.

Example. Five delivery times 30, 35, 28, 40 and 32 minutes have sample mean 165 / 5 = 33 minutes.

Related: Expected value, Standard error, Law of large numbers

The set of all possible outcomes of a random experiment, usually written as the Greek letter omega. Every event is a subset of it, and the probabilities of all its outcomes add up to 1. Defining it carefully is the first step in solving a probability problem.

Example. For one die roll the sample space is {1, 2, 3, 4, 5, 6}. For two coin tosses it is {HH, HT, TH, TT}.

Related: Outcome, Event, Probability axioms

A systematic error caused by choosing a sample in a way that makes some members of the population more or less likely to be included than others, so results do not represent the whole. Larger samples do not fix it.

Example. Surveying only people who reply to an app notification over-represents active users and misstates overall satisfaction.

Related: Random sampling, Selection bias, Survivorship bias

Drawing items one at a time and returning each to the pool before the next draw, so the same item can be chosen again and draws are independent. It is the basis of bootstrap resampling.

Example. Picking 5 names from a hat, putting each back after reading it, so a name may appear twice.

Related: Sampling without replacement, Bootstrap, Independence

Drawing items so that each one, once chosen, is not returned, so no item appears twice. Draws are dependent, but when the sample is small relative to the population the difference from replacement is negligible.

Example. Dealing 5 cards from a deck: each card taken changes the mix left in the deck.

Related: Sampling with replacement, Hypergeometric distribution, Dependent events

A sample in which every possible group of a given size from the population is equally likely to be chosen. It is the basic benchmark against which other sampling designs are compared.

Example. Drawing 50 roll numbers from a class list of 300 using a fair lottery draw.

Related: Random sampling, Stratified sampling, Sampling with replacement

A phenomenon in which a trend visible within several groups reverses or disappears when the groups are combined. It arises when group sizes differ and a hidden variable affects both the groups and the outcome, warning against careless aggregation.

Example. A treatment can have a higher success rate in each of two hospitals yet a lower overall rate if it is used mostly on the harder cases.

Related: Confounding variable, Marginal probability, Correlation vs causation

A measure of how lopsided a distribution is around its mean. Positive skew means a long tail to the right, negative skew means a long tail to the left, and zero suggests symmetry. It is based on the third standardised moment.

Example. Household incomes are usually right-skewed: most are modest, with a few very large ones stretching the right tail.

Related: Kurtosis, Moment, Median

The square root of the variance, giving spread in the same units as the variable itself. For bell-shaped data, about 68% of values lie within one standard deviation of the mean and about 95% within two.

Example. If delivery times average 30 minutes with a standard deviation of 5, most deliveries take between 25 and 35 minutes.

Related: Variance, Normal distribution, Expected value

The normal distribution with mean 0 and standard deviation 1. Any normal variable can be converted to it by subtracting its mean and dividing by its standard deviation, which lets one table or function serve every normal distribution.

Example. A value of 1.96 on this scale has about 97.5% of the probability below it.

Related: Normal distribution, Z-score, Cumulative distribution function

A distribution over the states of a Markov chain that stays unchanged after one more step. Under suitable conditions the chain converges to it regardless of the starting state, which is the idea behind PageRank and MCMC.

Example. A two-state chain with switching probabilities 0.1 and 0.2 settles at long-run shares of 2/3 and 1/3.

Related: Markov chain, Transition matrix, Markov chain Monte Carlo

A sampling design that first divides the population into non-overlapping groups called strata, then draws a random sample from each. It guarantees representation of every group and can reduce variability.

Example. A survey of 1,000 shoppers picks 400 from Tier-1 cities, 400 from Tier-2 and 200 from Tier-3, matching their share of the market.

Related: Cluster sampling, Simple random sample, Sampling bias

A distortion from looking only at cases that made it through some selection process while ignoring those that dropped out, leading to overly optimistic conclusions about what causes success. It is a special case of selection bias in which failures leave no visible trace.

Example. Studying only startups that are still operating to find success factors overlooks the many that failed with the same habits.

Related: Sampling bias, Selection bias, Base rate

A design that picks every k-th item from an ordered list after a random starting point. It is simple to carry out but can be misleading if the list has a repeating pattern matching k.

Example. From a list of 10,000 customers, start at a random position between 1 and 20 and take every 20th one.

Related: Simple random sample, Cluster sampling, Sampling bias

A square table in which entry (i, j) gives the probability of moving from state i to state j in one step of a Markov chain. Each row sums to 1, and multiplying it by itself gives multi-step transitions.

Example. If a customer active this month stays active with probability 0.9 and churns with 0.1, that forms one row of the table.

Related: Markov chain, Stationary distribution, Conditional probability

A distribution in which every value in a range is equally likely. For a continuous range from a to b, the density is constant at 1 / (b - a). Random number generators usually produce it first.

Example. A random number between 0 and 1 is uniform. A fair die is discrete uniform over 1 to 6.

Related: Random number generator, Probability density function, Normal distribution

The event that at least one of two or more given events occurs, written A or B. It contains every outcome that lies in any of the events. Its probability is found with the addition rule.

Example. With a die, the union of 'even' {2,4,6} and 'greater than 4' {5,6} is {2,4,5,6}.

Related: Intersection of events, Addition rule, Event

A measure of how spread out a random variable is around its mean: Var(X) = E[(X - mean)^2] = E[X^2] - (E[X])^2. Larger variance means values are more scattered and less predictable. Its units are the square of the variable's own units.

Example. Two shops both average 100 sales a day; one always sells 95 to 105, the other swings from 20 to 180. The second has far greater variance.

Related: Standard deviation, Expected value, Covariance

The number of standard deviations a value lies above or below the mean: z = (x - mean) / standard deviation. It puts values from different scales on a common footing and flags unusually large or small values.

Example. A student scoring 82 when the class mean is 70 and standard deviation is 6 has z = (82 - 70) / 6 = 2.

Related: Normal distribution, Standard deviation, Standardisation