Test p-value
The probability of seeing such a gap or a larger one if there is no effect: 2 × (1 − Φ(|z|)); shown as “<0.05%” when below the display limit; judge the effect size by the interval
- Formula
The probability of seeing such a gap or a larger one if there is no effect: 2 × (1 − Φ(|z|)); shown as “<0.05%” when below the display limit; judge the effect size by the interval- Unit
- %
- Aggregation rule
- Snapshot at a date
- Source system
- Website and app analytics; CRM, CDP and loyalty program
- Owning role
- Analyst (BI)
- Frequency
- Weekly
Passport
- Grain
- Test × metric
- At zero or no data
- A test with no data: “—”. Values below the display limit are shown as “<0.05%”.
- Limits
- It is not the probability that the hypothesis is true. Judge the effect size by the interval. Many comparisons and stopping early inflate false findings.
- Check example
- z statistic (standard deviations): 2.
2*(1-phi(Math.abs(z)))*100. Result: 4.55% - Related metrics
Version 1.0 · valid from 2026-11-16 · checked 2026-11-16
Related pages
- Term: p-value
- Calculation: A/B test on proportions: sample size, duration and reading the result
Definition
The probability of seeing such a gap or a larger one between variants if there is in fact no effect.
A small p does not prove the size of the effect and does not say B is better by so much: look at the confidence interval. Many checks in a row increase the share of false wins.