What Is a P-Value? Meaning, Examples and Common Mistakes
A p-value is the probability of data at least this extreme if the null hypothesis were true, not the chance the null is true. z-test and t-test examples.
A p-value is the probability of getting a result at least as extreme as the one you observed, assuming the null hypothesis (and every other assumption of the test) is true. If a sample of 100 has a mean of 103 and the null says the population mean is 100 with a known standard deviation of 15, the test statistic is z = 2.00 and the two-sided p-value is 0.0455. That does not mean there is a 4.55% chance that the null is true. It means that if the null were true, results this far from 100 in either direction would turn up in about 4.55% of samples of that size.
The ingredients
Every p-value needs a null hypothesis (here, the mean is 100), a test statistic that measures how far the data sits from it in standard-error units (a z-score, t-statistic or chi-square value), and the statistic’s reference distribution when the null is true (normal, Student’s t or chi-square). You also choose the tails: a two-sided p-value counts deviations in both directions, a one-sided one counts a single direction and must be chosen before seeing the data. “At least as extreme” is the easily forgotten part: the p-value is a tail area, not the probability of the exact result you saw.
What a p-value is not
| Common reading | Why it is wrong |
|---|---|
| “There is a 4.55% chance the null is true” (or “a 95.45% chance the effect is real”). | The p-value is computed assuming the null is true; it describes data, not hypotheses. The probability of a hypothesis needs prior information and a different method. |
| “Only 4.55% of significant results like this are false positives.” | The share of significant findings that are false depends on how many tested hypotheses were true to begin with and how much power the tests had. The p-value alone cannot tell you. |
| “A small p means a large effect.” | P-values mix effect size with sample size (see below). |
| “p > 0.05 proves there is no effect.” | A large p-value means the data are compatible with the null, not that the null is shown to be true. The test may simply have too little data. |
A z-test illustration
Null mean 100, known standard deviation 15, sample mean 103 each time. The standard error is 15 ÷ √n and z = 3 ÷ standard error.
| Sample size n | Standard error | z | Two-sided p | One-sided p |
|---|---|---|---|---|
| 25 | 3.00 | 1.00 | 0.3173 | 0.1587 |
| 100 | 1.50 | 2.00 | 0.0455 | 0.0228 |
| 400 | 0.75 | 4.00 | 0.000063 | 0.000032 |
The difference is the same 3 points each time; only the amount of data changes, and the p-value goes from unremarkable to tiny.
A t-test illustration
When the standard deviation is estimated from the sample, the reference distribution becomes Student’s t, with n − 1 degrees of freedom. Twelve measurements: 102, 98, 107, 110, 95, 104, 101, 108, 99, 106, 103 and 109, tested against a null mean of 100.
- Sample mean 103.5, sample standard deviation 4.70, standard error 4.70 ÷ √12 = 1.357.
- t = 3.5 ÷ 1.357 = 2.58 with 11 degrees of freedom.
- Two-sided p = 0.0256 (one-sided 0.0128).
- The 95% confidence interval for the mean is 100.51 to 106.49, just excluding 100, matching p below 0.05.
Treating that 2.58 as a z-score would give p = 0.0099, about 2.6 times too small. With few observations the t distribution has heavier tails, so the 5% cut-off is 2.20 here, not 1.96.
Alpha thresholds
The significance level, alpha, is the false-positive rate you accept when the null is true. Fix it before looking at the data; a result is “statistically significant” if p falls below it.
| Alpha | Two-sided critical z | One-sided critical z |
|---|---|---|
| 0.10 | 1.645 | 1.282 |
| 0.05 | 1.960 | 1.645 |
| 0.01 | 2.576 | 2.326 |
| 0.001 | 3.291 | 3.090 |
In 20,000 simulated samples of 30 values drawn from a population whose mean really is 100, a t-test at alpha = 0.05 flagged 5.05% of them as significant, and 1.09% at alpha = 0.01. Rejecting the null at 0.05 is wrong about one time in twenty when the null is true; no threshold removes this. The 0.05 line is a convention: p = 0.049 and p = 0.051 are not meaningfully different.
The chi-square version
For counts the statistic is chi-square. A die rolled 120 times gives 25, 17, 15, 23, 24 and 16 across faces 1 to 6. A fair die predicts 20 each, so chi-square is 5.0 with 5 degrees of freedom and p = 0.4159, well short of the 5% critical value of 11.07. That is compatible with a fair die, not proof of one.
Effect size: significant is not important
A tiny difference becomes significant given enough data. All three rows below have z = 2.00 and p = 0.0455, but the standardised effect size (the difference measured in standard deviations) shrinks as the sample grows:
| Sample size | Effect in standard deviations | z | Two-sided p |
|---|---|---|---|
| 100 | 0.2 | 2.00 | 0.0455 |
| 10,000 | 0.02 | 2.00 | 0.0455 |
| 1,000,000 | 0.002 | 2.00 | 0.0455 |
A modest effect of 0.2 with 10,000 observations gives p near 5.5 × 10⁻⁸⁹. Report the effect and a confidence interval beside the p-value.
P-hacking in one paragraph
If you run many tests and report only the ones below 0.05, you will find “significant” results in pure noise. With 20 independent tests of true nulls at alpha = 0.05, the chance of at least one p below 0.05 is 1 − 0.95²⁰ = 64.15%; with 100 tests it is 99.41%. Trying outcomes, subgroups or exclusions until one crosses the line has the same effect. Defences: state the analysis before seeing the data, count every test, and adjust. The simplest correction divides alpha by the number of tests, so 20 tests use 0.0025 each.
Try it
Enter a z or t statistic into the p-value calculator for one- and two-sided p-values. To turn a raw value, mean and standard deviation into a z-score first, use the z-score calculator, and for observed counts against expectations, the chi-square calculator. For the spread statistics behind standard errors, see mean, median and mode. All values above were computed with SciPy; the simulation uses a fixed seed.
Updated 2026-11-04