Empirical and Theoretical Probability

Theoretical probability is calculated mathematically from the structure of an experiment. Empirical probability (experimental probability) comes from observed data. A key result is that as sample size grows, empirical distributions tend toward theoretical distributions — making large random samples representative of the population.

Why Sample Size Matters

A small sample may produce frequencies that differ substantially from the theoretical distribution — this is expected variation, not evidence of bias. With a large, unbiased sample, the observed distribution closely mirrors the theoretical one. This principle underpins statistical inference: we use samples to estimate population parameters.

Unbiased Samples

For a sample to represent a population reliably, it must be unbiased — every member of the population must have an equal chance of being selected (random sampling). A biased sample systematically over- or under-represents part of the population, distorting empirical probabilities.

Comparing Empirical and Theoretical

	Theoretical	Empirical
Source	Mathematical structure	Observed data
When to use	Equally likely outcomes known	Unknown or unequal outcomes
Reliability	Exact (given assumptions)	Improves with sample size
Example	P(heads) = 0.5 for fair coin	380/1000 from experiment

Key Takeaways

Empirical probability: from observed data; approaches theoretical with large unbiased samples.
Theoretical probability: from equally-likely-outcome structure; exact under assumptions.
Larger unbiased samples give more reliable empirical estimates.
Biased samples distort empirical probabilities — source of sampling error.
The Law of Large Numbers: relative frequency → theoretical probability as \( n\to\infty \).