Syntha Lab RU

Bayesian estimate of a rate: buyout or conversion of a new style from a small sample

Combines your prior estimate of a rate (by category or past styles) with a small number of observations: shows the posterior rate, the interval, the probability of exceeding a target, the weight of the data against the prior, a comparison with the ordinary Wilson interval and the expected range of successes in the next trials.

Marketing budget and testing impact For whom: Category managers and analysts assessing a new style, a new store or a channel from a small sample.

Calculate on your data →

At a glance

What question does it answer?
What the buyout or conversion rate is when data are scarce and with what probability it exceeds the plan.
What data is needed?
The prior rate and its strength: from the category or past styles.
How to read the result?
Example on the default values (a prior rate of 70% with a strength of 30, 24 of 40 bought out): the data give 60%, the posterior rate is 64.3%, the interval from 52.8 to 75%. The probability above the 65% target is 46%. Out of 100 new orders 64 are expected. Example on the default values: Rate in the data: 60%; Posterior rate (mean): 64.3%; Posterior rate (median): 64.4%; Interval: lower bound: 52.8%. The figures are illustrative: enter your own to see your result.

Why and how to reason

A new model has been released, 40 units sold, 24 redeemed, that is 60%. Can these 60% be trusted? On a small sample they fluctuate easily: two or three orders one way or the other shift the figure by several points. But you also have prior knowledge: the category usually redeems at 70%. The Bayesian approach combines both sources. Prior knowledge is written as a conditional number of observations: 70% with strength 30 is like 21 successes and 9 failures (Beta(21, 9)). The data add 24 successes and 16 failures: Beta(45, 25). The mean is 45 ÷ 70 = 64.3%. The weight of data is 40 ÷ 70 = 57%: the more data, the smaller the weight of prior knowledge. The interval of 52.8 to 75.0% is narrower than the Wilson interval without prior knowledge (44.6 to 73.7%). The probability that the share is above the 65% target is 46%.

When to use it

Use it for new models, channels and tests with a small sample when there is justified prior knowledge (category, past models). Do not use it if the prior is doubtful or the model is fundamentally new: a strong but wrong prior gives false confidence. Assumptions: observations are independent, the share is constant, the prior estimate and strength are set by you, the strength does not exceed the experience of similar models.

How to read the result

Look at the weight of data and the comparison of intervals. A weight of 57% means data and prior knowledge contribute comparably: if this seems wrong to you, change the strength of the prior. The interval is narrower than Wilson’s, but that is a consequence of the assumption, not magic. The probability of being above the target (46%) answers “is it worth waiting for more data”: about half means it is too early to decide. The range of successes in the next 100 trials (52 to 76) helps plan buying. Decision: if the probability of the target is below 30% or above 80%, you can act, otherwise gather data.

Common mistakes

Taking the strength of the prior equal to the volume of data “to be confident”, using a prior share from another category, treating orders of one customer as independent, stopping at the first probability above 90%. Always show the interval without the prior as well so the reader sees the difference. Do not change the prior after seeing the data: that turns the method into fitting.

What data is needed?

  • The prior rate and its strength: from the category or past styles.
  • Trials and successes.
  • The target rate and the number of future trials.

Inputs

FieldUnitValue in the exampleMeaning
Prior rate%70What you believed before the data: the average rate for the category or past styles. The example value is illustrative.
Prior strengthobservations30How many observations your belief is worth. The larger, the slower the data change it. The weight of the data is counted from this number and the number of trials. The example value is illustrative.
Trialsitems40How many orders, impressions or visits have occurred. The example value is illustrative.
Successesitems24How many of them were bought out or bought. Not above the number of trials.
Target%65The rate above which the style is considered a success (for example the planned buyout). Your threshold.
Future trialsitems100For forecasting the number of successes, for example orders of the next lot. Not above 10,000.
Interval level95%The probability with which the rate lies inside the interval. A higher level: a wider interval.

How it is calculated

StepFormula
PriorBeta(a0, b0): a0 = rate × strength, b0 = (1 − rate) × strength
After the dataBeta(a0 + successes, b0 + trials − successes)
Meana ÷ (a + b)
Intervalbeta-distribution quantiles at (1 − level) ÷ 2 and 1 − (1 − level) ÷ 2
Above the target1 − I_target(a, b), I the incomplete beta function
Forecastbeta-binomial: P(k) = C(m, k) × B(k + a, m − k + b) ÷ B(a, b)

Worked numbers

Default values from the app; in the interactive version you enter your own.

Rate in the data
60 %Successes ÷ trials: without the prior.
Posterior rate (mean)
64.3 %The data pulled towards the prior the more, the fewer the trials.
Posterior rate (median)
64.4 %
Interval: lower bound
52.8 %
Interval: upper bound
75 %
Interval width, p.p.
22.2
Wilson interval without a prior: lower bound
44.6 %
Wilson interval without a prior: upper bound
73.7 %
Probability that the rate is above the 65% target
46 %
Weight of the data against the prior
57.1 %Trials ÷ (trials + prior strength).
Expected successes out of 100 new
64
Range of successes 90%: lower bound
52
Range of successes 90%: upper bound
76
What you thought of the rate before and after the data
Line chart: Before the data: 0 → 0; After the data: 0 → 000.020.040.060.0820406080targetBefore the data, 1: 0Before the data, 99: 0After the data, 1: 0After the data, 99: 0DensityRate, %
  • Before the data
  • After the data
Data as a table
Rate, %Before the dataAfter the data
100
200
300
400
500
600
700
800
900
1000
1100
1200
1300
1400
1500
1600
1700
1800
1900
2000
2100
2200
2300
2400
2500
2600
2700
2800
2900
3000
3100
3200
3300
3400
3500
3600
3700
3800
3900
4000
4100
4200
430.0010
440.0010
450.0010
460.0010.001
470.0020.001
480.0020.002
490.0030.002
500.0030.004
510.0040.005
520.0050.007
530.0070.01
540.0080.014
550.010.019
560.0120.024
570.0140.03
580.0160.037
590.0190.044
600.0220.051
610.0250.057
620.0280.063
630.0310.067
640.0340.069
650.0370.07
660.040.068
670.0420.064
680.0440.059
690.0460.052
700.0470.045
710.0480.037
720.0480.03
730.0470.023
740.0460.017
750.0440.012
760.0410.008
770.0380.005
780.0340.003
790.0310.002
800.0270.001
810.0230.001
820.0190
830.0150
840.0120
850.0090
860.0070
870.0050
880.0030
890.0020
900.0010
910.0010
9200
9300
9400
9500
9600
9700
9800
9900

Prior and posterior distribution

DistributionabMean, %
Before the data21970
After the data452564.3

The data give 60 %, the prior 70 %, the result 64.3 %: the data weigh 57 %. The probability of exceeding the target is 46 %.

The result is as good as the prior is honest: an overconfident one (large strength) drowns the data, a too weak one gives a wide interval. Peeking at the data and stopping at a convenient result distorts the probability. Footwear, accessories and children’s have their own prior rate.

Nuances and limits

  • Setting the prior with a huge strength: the data will change nothing.
  • Stopping data collection when the result is liked: the probability is distorted.
  • Confusing the probability above the target with the rate you will “surely” get.

Terms used

Calculate on your data →