Bayesian estimate of a rate: buyout or conversion of a new style from a small sample
Combines your prior estimate of a rate (by category or past styles) with a small number of observations: shows the posterior rate, the interval, the probability of exceeding a target, the weight of the data against the prior, a comparison with the ordinary Wilson interval and the expected range of successes in the next trials.
At a glance
- What question does it answer?
- What the buyout or conversion rate is when data are scarce and with what probability it exceeds the plan.
- What data is needed?
- The prior rate and its strength: from the category or past styles.
- How to read the result?
- Example on the default values (a prior rate of 70% with a strength of 30, 24 of 40 bought out): the data give 60%, the posterior rate is 64.3%, the interval from 52.8 to 75%. The probability above the 65% target is 46%. Out of 100 new orders 64 are expected. Example on the default values: Rate in the data: 60%; Posterior rate (mean): 64.3%; Posterior rate (median): 64.4%; Interval: lower bound: 52.8%. The figures are illustrative: enter your own to see your result.
Why and how to reason
A new model has been released, 40 units sold, 24 redeemed, that is 60%. Can these 60% be trusted? On a small sample they fluctuate easily: two or three orders one way or the other shift the figure by several points. But you also have prior knowledge: the category usually redeems at 70%. The Bayesian approach combines both sources. Prior knowledge is written as a conditional number of observations: 70% with strength 30 is like 21 successes and 9 failures (Beta(21, 9)). The data add 24 successes and 16 failures: Beta(45, 25). The mean is 45 ÷ 70 = 64.3%. The weight of data is 40 ÷ 70 = 57%: the more data, the smaller the weight of prior knowledge. The interval of 52.8 to 75.0% is narrower than the Wilson interval without prior knowledge (44.6 to 73.7%). The probability that the share is above the 65% target is 46%.
When to use it
Use it for new models, channels and tests with a small sample when there is justified prior knowledge (category, past models). Do not use it if the prior is doubtful or the model is fundamentally new: a strong but wrong prior gives false confidence. Assumptions: observations are independent, the share is constant, the prior estimate and strength are set by you, the strength does not exceed the experience of similar models.
How to read the result
Look at the weight of data and the comparison of intervals. A weight of 57% means data and prior knowledge contribute comparably: if this seems wrong to you, change the strength of the prior. The interval is narrower than Wilson’s, but that is a consequence of the assumption, not magic. The probability of being above the target (46%) answers “is it worth waiting for more data”: about half means it is too early to decide. The range of successes in the next 100 trials (52 to 76) helps plan buying. Decision: if the probability of the target is below 30% or above 80%, you can act, otherwise gather data.
Common mistakes
Taking the strength of the prior equal to the volume of data “to be confident”, using a prior share from another category, treating orders of one customer as independent, stopping at the first probability above 90%. Always show the interval without the prior as well so the reader sees the difference. Do not change the prior after seeing the data: that turns the method into fitting.
What data is needed?
- The prior rate and its strength: from the category or past styles.
- Trials and successes.
- The target rate and the number of future trials.
Inputs
| Field | Unit | Value in the example | Meaning |
|---|---|---|---|
| Prior rate | % | 70 | What you believed before the data: the average rate for the category or past styles. The example value is illustrative. |
| Prior strength | observations | 30 | How many observations your belief is worth. The larger, the slower the data change it. The weight of the data is counted from this number and the number of trials. The example value is illustrative. |
| Trials | items | 40 | How many orders, impressions or visits have occurred. The example value is illustrative. |
| Successes | items | 24 | How many of them were bought out or bought. Not above the number of trials. |
| Target | % | 65 | The rate above which the style is considered a success (for example the planned buyout). Your threshold. |
| Future trials | items | 100 | For forecasting the number of successes, for example orders of the next lot. Not above 10,000. |
| Interval level | 95% | The probability with which the rate lies inside the interval. A higher level: a wider interval. |
How it is calculated
| Step | Formula |
|---|---|
| Prior | Beta(a0, b0): a0 = rate × strength, b0 = (1 − rate) × strength |
| After the data | Beta(a0 + successes, b0 + trials − successes) |
| Mean | a ÷ (a + b) |
| Interval | beta-distribution quantiles at (1 − level) ÷ 2 and 1 − (1 − level) ÷ 2 |
| Above the target | 1 − I_target(a, b), I the incomplete beta function |
| Forecast | beta-binomial: P(k) = C(m, k) × B(k + a, m − k + b) ÷ B(a, b) |
Worked numbers
Default values from the app; in the interactive version you enter your own.
- Rate in the data
- 60 %Successes ÷ trials: without the prior.
- Posterior rate (mean)
- 64.3 %The data pulled towards the prior the more, the fewer the trials.
- Posterior rate (median)
- 64.4 %
- Interval: lower bound
- 52.8 %
- Interval: upper bound
- 75 %
- Interval width, p.p.
- 22.2
- Wilson interval without a prior: lower bound
- 44.6 %
- Wilson interval without a prior: upper bound
- 73.7 %
- Probability that the rate is above the 65% target
- 46 %
- Weight of the data against the prior
- 57.1 %Trials ÷ (trials + prior strength).
- Expected successes out of 100 new
- 64
- Range of successes 90%: lower bound
- 52
- Range of successes 90%: upper bound
- 76
- Before the data
- After the data
Data as a table
| Rate, % | Before the data | After the data |
|---|---|---|
| 1 | 0 | 0 |
| 2 | 0 | 0 |
| 3 | 0 | 0 |
| 4 | 0 | 0 |
| 5 | 0 | 0 |
| 6 | 0 | 0 |
| 7 | 0 | 0 |
| 8 | 0 | 0 |
| 9 | 0 | 0 |
| 10 | 0 | 0 |
| 11 | 0 | 0 |
| 12 | 0 | 0 |
| 13 | 0 | 0 |
| 14 | 0 | 0 |
| 15 | 0 | 0 |
| 16 | 0 | 0 |
| 17 | 0 | 0 |
| 18 | 0 | 0 |
| 19 | 0 | 0 |
| 20 | 0 | 0 |
| 21 | 0 | 0 |
| 22 | 0 | 0 |
| 23 | 0 | 0 |
| 24 | 0 | 0 |
| 25 | 0 | 0 |
| 26 | 0 | 0 |
| 27 | 0 | 0 |
| 28 | 0 | 0 |
| 29 | 0 | 0 |
| 30 | 0 | 0 |
| 31 | 0 | 0 |
| 32 | 0 | 0 |
| 33 | 0 | 0 |
| 34 | 0 | 0 |
| 35 | 0 | 0 |
| 36 | 0 | 0 |
| 37 | 0 | 0 |
| 38 | 0 | 0 |
| 39 | 0 | 0 |
| 40 | 0 | 0 |
| 41 | 0 | 0 |
| 42 | 0 | 0 |
| 43 | 0.001 | 0 |
| 44 | 0.001 | 0 |
| 45 | 0.001 | 0 |
| 46 | 0.001 | 0.001 |
| 47 | 0.002 | 0.001 |
| 48 | 0.002 | 0.002 |
| 49 | 0.003 | 0.002 |
| 50 | 0.003 | 0.004 |
| 51 | 0.004 | 0.005 |
| 52 | 0.005 | 0.007 |
| 53 | 0.007 | 0.01 |
| 54 | 0.008 | 0.014 |
| 55 | 0.01 | 0.019 |
| 56 | 0.012 | 0.024 |
| 57 | 0.014 | 0.03 |
| 58 | 0.016 | 0.037 |
| 59 | 0.019 | 0.044 |
| 60 | 0.022 | 0.051 |
| 61 | 0.025 | 0.057 |
| 62 | 0.028 | 0.063 |
| 63 | 0.031 | 0.067 |
| 64 | 0.034 | 0.069 |
| 65 | 0.037 | 0.07 |
| 66 | 0.04 | 0.068 |
| 67 | 0.042 | 0.064 |
| 68 | 0.044 | 0.059 |
| 69 | 0.046 | 0.052 |
| 70 | 0.047 | 0.045 |
| 71 | 0.048 | 0.037 |
| 72 | 0.048 | 0.03 |
| 73 | 0.047 | 0.023 |
| 74 | 0.046 | 0.017 |
| 75 | 0.044 | 0.012 |
| 76 | 0.041 | 0.008 |
| 77 | 0.038 | 0.005 |
| 78 | 0.034 | 0.003 |
| 79 | 0.031 | 0.002 |
| 80 | 0.027 | 0.001 |
| 81 | 0.023 | 0.001 |
| 82 | 0.019 | 0 |
| 83 | 0.015 | 0 |
| 84 | 0.012 | 0 |
| 85 | 0.009 | 0 |
| 86 | 0.007 | 0 |
| 87 | 0.005 | 0 |
| 88 | 0.003 | 0 |
| 89 | 0.002 | 0 |
| 90 | 0.001 | 0 |
| 91 | 0.001 | 0 |
| 92 | 0 | 0 |
| 93 | 0 | 0 |
| 94 | 0 | 0 |
| 95 | 0 | 0 |
| 96 | 0 | 0 |
| 97 | 0 | 0 |
| 98 | 0 | 0 |
| 99 | 0 | 0 |
Prior and posterior distribution
| Distribution | a | b | Mean, % |
|---|---|---|---|
| Before the data | 21 | 9 | 70 |
| After the data | 45 | 25 | 64.3 |
The data give 60 %, the prior 70 %, the result 64.3 %: the data weigh 57 %. The probability of exceeding the target is 46 %.
The result is as good as the prior is honest: an overconfident one (large strength) drowns the data, a too weak one gives a wide interval. Peeking at the data and stopping at a convenient result distorts the probability. Footwear, accessories and children’s have their own prior rate.
Nuances and limits
- Setting the prior with a huge strength: the data will change nothing.
- Stopping data collection when the result is liked: the probability is distorted.
- Confusing the probability above the target with the rate you will “surely” get.