P-Hacker Simulator
Chase a star. Then count the false alarms.
How many chances do you want?
Each outcome supplies an independent Uniform(0,1) p-value under a true null. This is a simulation of valid null tests, not a fitted regression or a real dataset.
No star this time. After adjustment, this family has no rejection at 0.05.
Experiment 1 · reproducible seed 20268933. Asterisk: unadjusted p < 0.05. The CSV keeps full precision.
The cost of looking.
For m independent null tests at 0.05, the chance of at least one rejection is 1 − 0.95ᵐ. Bonferroni compares each p-value with 0.05/m and controls family-wise error even without independence. The exact curve here assumes independence.
Plan primary outcomes, disclose every analysis and treat unplanned searches as exploration. A small p-value alone does not establish a causal effect.
Read the method: J-PAL · Data analysis ↗Read the method: American Statistical Association · Statement on p-values (2016) ↗