Syntha Lab RU

Bayesian estimates of conversion and buyout: small samples, prior distributions, shrinkage and comparison with the frequentist approach

How to estimate a share (conversion, buyout, returns) and a sales rate when data is scarce: a new store, a new model, three days of sales. The Bayesian approach combines experience (the distribution across the chain or category) with new observations, gives an interval as a probability and a direct answer to “what is the probability that variant B is better than A”. The decision: what to treat as the estimate while data is scarce and when a decision can already be taken.

Math and analytics Brand and retail Verified: 2026-11-14

Open interactively →

Why this document

The analyst and the manager read this when data is scarce: a new store, a new model, three days of sales. A share computed from ten orders jumps by chance, and a conclusion like a 30% conversion is worth nothing. A Bayesian estimate combines such observations with experience across the network or category and pulls extreme values toward the mean, so a small sample gives neither false records nor false failures. As data accumulates, the weight of observations grows and the estimate converges with the ordinary one. That way a decision to buy or close is not built on noise.

At a glance

What is this document and why is it needed?
How to estimate a share (conversion, buyout, returns) and a sales rate when data is scarce: a new store, a new model, three days of sales. The Bayesian approach combines experience (the distribution across the chain or category) with new observations, gives an interval as a probability and a direct answer to “what is the probability that variant B is better than A”. The decision: what to treat as the estimate while data is scarce and when a decision can already be taken.
Who owns the document and how often is it rebuilt?
A sales, e-commerce or CRM analyst; prior distributions are reviewed once a season, estimates are updated as data comes in; read by category managers, marketing and store managers.
What mistakes are the most common?
A store ranking by raw shares at different n → random small samples on top → shrunk estimates and intervals next to the ranking. Too strong a prior from another category → the estimate does not respond to data → a prior from the own group and a check on a weak prior. The prior and its strength are hidden → the estimate cannot be reproduced or challenged → the prior version and parameters are stored with the estimate.

For whom

Brand
A brand estimates site conversion and the buyout of a new collection from the first days and small partners, using the history of past collections as a prior.
Retail
A retailer estimates the conversion of new and small stores, returns of a new model and the sales rate of a new item from the first days, relying on the distribution across the chain and category.

What it consists of

When the frequentist estimate fails

  • The share x/n of a small sample is noisy: in the example a store with 350 visitors showed 16.6% against a true 14.7%, and a store with 2200 visitors 9.5% against 7.9%. A ranking by raw shares can put chance successes of small samples on top.
  • The Wilson confidence interval honestly shows uncertainty but does not use what we know in advance: a chain store’s conversion cannot be 40% or 0.1%. A Bayesian estimate adds that knowledge as a prior and gives a stable answer already on the first observations.
  • A prior is not a guess but a distribution estimated from data: from chain stores, past collections, analogs. The less data and the narrower the prior the more it matters; with a large sample the prior ceases to matter.

The beta-binomial model

  • Conversion p has a prior Beta(α, β) with mean α/(α + β) and strength κ = α + β (how many “virtual observations” it is worth). After x purchases out of n visitors the posterior is Beta(α + x, β + n − x) and its mean (x + α)/(n + κ) is a weighted average of the raw share x/n and the prior.
  • The prior is estimated from chain stores by the method of moments with a correction for binomial noise: in the example average conversion is 8.5%, strength κ = 159 (α = 13.5, β = 145.8). The weight of the raw share n/(n + κ) is 69% for a store with 350 visitors and 99% for a store with 20,000.
  • The result of shrinkage: the mean squared error of estimating true conversion is 0.65·10⁻⁴ for raw shares and 0.32·10⁻⁴ for Bayesian ones, and for the five smallest stores 1.37·10⁻⁴ against 0.62·10⁻⁴. Of the three best stores by true conversion the raw ranking finds 3, the Bayesian 3.
  • Brand: the prior for a new partner or channel comes from the distribution over partners of the same type and size, not from the brand average across all sales; large partners make the average unrepresentative.
  • Retailer: the prior for a new store comes from stores of the same format, city and area; a mall and a street give different distributions and must not be mixed.

A new store and intervals

  • A new store: 7 purchases out of 120 visitors, a raw share of 5.8%. The Bayesian estimate with the chain prior is 7.3%, the 95% interval 4.6-10.7%; with a flat prior the interval is 2.9-11.6%, Wilson’s 2.9-11.6%. The informative prior narrows the interval by 30%.
  • A credible interval is a direct statement: with 95% probability the true conversion lies within these limits under the chosen model. A frequentist confidence interval speaks of the share of such intervals covering the truth under repeated sampling; for decisions the first reading is handier but it holds only if the prior is justified.
  • Prior sensitivity: at the same raw share of 5.8% and n = 20 the estimate ranges from 6.7% (a weak prior) to 9.8% (flat), at n = 2,500 the spread is 0.15 pp. Conclusion: at small n the prior must be shown and justified, and at large n its influence disappears.

Comparing variants A and B

  • Example: A 40 of 500 (8%), B 55 of 520 (10.6%). The frequentist answer: p-value 0.157, so at α = 5% the difference is not proven. The Bayesian answer: the probability that B is better than A is 92%, the expected loss if B is chosen 0.07 pp of conversion.
  • The two answers are consistent: there is too little data for “B is better with 95% certainty” but more than nine chances in ten point to B. The decision is taken by the cost of a mistake: if launching B is cheap and reversible this is enough, otherwise wait for data; the probability threshold and the expected loss threshold are set before the test.
  • Brand: the roll-out threshold for a D2C page is low because rollback is cheap; for a price or partner terms change the threshold is higher because rollback is costly and visible to partners.
  • Retailer: the threshold for a chain-wide promotion is higher than for a pilot in three stores: the cost of a mistake grows with reach, and the Bayesian estimate from the pilot is updated as sales come in.
  • The Bayesian approach does not remove peeking and multiple comparisons problems entirely: the stopping rule and prior are fixed in advance and the number of variants is accounted for through the prior and thresholds, as in the frequentist approach (the A/B tests document).

Buyout of a new model and a new item forecast from the first days

  • Returns of a new model: 14 of 40 (35%) with a category prior of mean 22% and strength 60. The posterior mean is 27.2%, the probability that returns exceed the 30% threshold is 26%. The decision is not “returns 35%, withdraw” but “the probability of exceeding the threshold is about a quarter, continue and recompute at 100 orders”.
  • The sales rate of a new item (units a day) is modeled by a gamma distribution and the number of sales by a Poisson one. The analog prior Gamma(6, 1.5) with a mean of 4 units a day and an observation of 14 units in 3 days (a raw rate of 4.7) give a posterior Gamma(20, 4.5) with a mean of 4.44 units a day.
  • The sales forecast for the next 14 days is built from the predictive distribution that accounts for both rate uncertainty and Poisson noise: a median of 61 units, an 80% interval of 43-84 units. The interval is what goes into the order and the reorder.

Brand and retailer: what is estimated in a Bayesian way

  • Brand: conversion of new channels and pages, the buyout of a new collection from the first batch, the return rate of a new partner; the prior comes from past collections and channels.
  • Retailer: conversion of new and small stores, the share of sales by discount card, returns of a new model and the rate of a new item; the prior comes from the distribution by chain, format and category.
  • Brand: D2C page and price variants are compared by the probability of “B better than A” and the expected loss, and the roll-out decision is tied to the cost of a mistake.
  • Retailer: a ranking of stores by conversion and receipt is built from shrunk estimates, not raw shares, so that bonuses and reviews are not given for the chance of a small sample.
  • Brand: the decision to reorder fabric from the first weeks of sales relies on a predictive interval, not a point forecast.
  • Retailer: the decision to reallocate a new item between stores from the first days relies on the posterior rate with an interval: one day without sales in a store is no reason to pull it.

Where the data comes from

Data sourceFields
Counters and receiptsVisitors and purchases by store and week, orders and returns by model, daily sales for a new item; the number of observations n and events x.
History for the priorConversion and returns by chain stores, past collections and analogs, sales rate of analogs by day of life.
Decision thresholdsThe acceptable return share, the “B better than A” probability threshold, the acceptable expected loss, the cost of a launch error and rollback.

How to build it step by step

  1. Step 1. Define what is estimated (a share or a rate) and the model: beta-binomial for shares, gamma-Poisson for a sales rate.
  2. Brand: Step 2. Assemble the history of past collections and channels (buyout, conversion) and estimate the prior by the method of moments; for a new channel take the nearest by audience.
  3. Retailer: Step 3. Assemble chain store conversions and estimate the prior by format and region; the prior strength is no larger than the visitors of a typical store month.
  4. Step 4. Estimate the mean level m and the between-object variance τ² with a correction for binomial noise; obtain α and β.
  5. Step 5. For each object compute the posterior Beta(α + x, β + n − x), the mean and a 95% credible interval; show n and the raw share next to them.
  6. Step 6. Check the prior for stability: recompute with a flat and a weak prior; if the decision changes the data is insufficient and more must be collected.
  7. Brand: Step 7. To compare variants compute the probability of “B better than A” and the expected loss; take the roll-out decision by thresholds set in advance and the cost of rollback.
  8. Retailer: Step 8. Build a store ranking from posterior means and intervals; do not reward or penalize stores whose intervals overlap.
  9. Step 9. For a new item build a predictive sales interval for the coming weeks and hand it to the order and reorder with the re-estimation point.
  10. Step 10. Update the prior once a season from new data and store the prior version with the estimate so that decisions can be reproduced.

Formulas

MeasureFormulasNote
Posterior of a sharep | x, n ~ Beta(α + x, β + n − x); E[p | x, n] = (x + α) / (n + α + β)Prior Beta(α, β) with mean α/(α + β) and strength κ = α + β.
ShrinkageE[p | x, n] = w·(x/n) + (1 − w)·m, w = n / (n + κ), m = α / (α + β)w grows with n: with a large sample the estimate equals x/n.
Empirical Bayesm = Σx_i / Σn_i, τ̂² = max( Σn_i(p̂_i − m)²/Σn_i − m(1 − m)/n̄, ε ), κ̂ = m(1 − m)/τ̂² − 1Method of moments; n̄ is the average object sample size.
Credible interval[ q_(0.025), q_(0.975) ] quantiles of Beta(α + x, β + n − x)A probability of 95% under the chosen model; depends on the prior.
Probability B > A and lossP(B > A) = P(p_B > p_A), L_B = E[ max(p_A − p_B, 0) ]Computed by sampling from two beta distributions (simulation) or by an integral.
Gamma-Poisson modelλ ~ Gamma(a, r); x | λ ~ Poisson(λ·t); λ | x ~ Gamma(a + x, r + t)Rate mean a/r; a prediction for horizon H is obtained by sampling λ and then the number of sales.

Formulas in notation

Posterior mean
E[p | x, n] = (x + α) / (n + α + β)
A weighted average of the share and the prior.
Weight of data
w = n / (n + κ), κ = α + β
Grows with sample size.
Prior strength
κ̂ = m(1 − m) / τ̂² − 1
An estimate from the spread between objects.
Probability B > A
P(p_B > p_A); L_B = E[ max(p_A − p_B, 0) ]
Sampling from two beta distributions.
Gamma-Poisson
λ | x ~ Gamma(a + x, r + t)
The sales rate of a new item.

Worked example

Twelve stores: raw share, Bayesian estimate and true conversion

StoreVisitorsRaw, %Bayesian, %True, %
135016.61414.7
26003.54.53.9
39005.86.26.1
41,50015.815.115
52,2009.59.47.9
63,00013.413.112.9
74,5001211.912
86,0007.27.27.6
98,0009.79.79.3
1011,0008.68.69.1
1114,0006.26.26.2
1220,0007.87.88

Store conversions are generated in code (fixed seed) from Beta(7.2, 72.8); the prior is estimated from the same stores (m = 8.5%, κ = 159). For the five smallest stores the mean squared error of the Bayesian estimate is 0.62·10⁻⁴ against 1.37·10⁻⁴ for the raw share. The data is illustrative.

Figures

A new store: prior and posterior of conversion
Line chart: Chain prior: 0 → 0; Posterior with chain prior: 0 → 0; Posterior with flat prior: 0 → 00510152025300.13.67.110.514.117.621.124.6Chain prior, 0.1: 0Chain prior, 25.1: 0Posterior with chain prior, 0.1: 0Posterior with chain prior, 25.1: 0Posterior with flat prior, 0.1: 0Posterior with flat prior, 25.1: 0density
  • Chain prior
  • Posterior with chain prior
  • Posterior with flat prior
Data as a table
MeasureChain priorPosterior with chain priorPosterior with flat prior
0.1000
0.5000
1.1000
1.6000.3
2.1001.1
2.6002.7
3.10.20.15.3
3.60.70.68.6
4.11.61.912
4.63.34.815.1
5.15.79.417.3
5.68.615.218.4
6.111.720.818.5
6.614.624.717.6
7.116.92616.1
7.618.224.614.1
8.118.521.212
8.617.816.89.9
916.412.47.9
9.614.48.56.2
10.112.25.54.7
10.5103.43.5
11.17.922.6
11.66.11.11.9
124.60.61.3
12.63.30.30.9
13.12.40.10.6
13.61.60.10.4
14.11.100.3
14.60.700.2
15.10.500.1
15.60.300.1
16.10.200.1
16.60.100
17.10.100
17.6000
18.1000
18.6000
19.1000
19.6000
20.1000
20.6000
21.1000
21.6000
22.1000
22.6000
23.1000
23.6000
24.1000
24.6000
25.1000

A new store: prior and posterior of conversion. The chain prior is concentrated near 8.5%; after 120 visitors and 7 purchases the posterior is shifted to 7.3% and narrower than the flat one: an interval of 4.6-10.7%. A flat prior gives a wide curve peaking at the raw share of 5.8%. Axis: conversion, %.

Twelve stores: raw share, Bayesian estimate and truth
Bar chart: 1 (350) 16.6, 2 (600) 3.5, 3 (900) 5.8, 4 (1,500) 15.8, 5 (2,200) 9.5, 6 (3,000) 13.4051015201 (350)1 (350)Raw share: 16.616.6Bayesian estimate: 1414True: 14.714.72 (600)2 (600)Raw share: 3.53.5Bayesian estimate: 4.54.5True: 3.93.93 (900)3 (900)Raw share: 5.85.8Bayesian estimate: 6.26.2True: 6.16.14 (1,500)4 (1,500)Raw share: 15.815.8Bayesian estimate: 15.115.1True: 15155 (2,200)5 (2,200)Raw share: 9.59.5Bayesian estimate: 9.49.4True: 7.97.96 (3,000)6 (3,000)Raw share: 13.413.4Bayesian estimate: 13.113.1True: 12.912.97 (4,500)7 (4,500)Raw share: 1212Bayesian estimate: 11.911.9True: 12128 (6,000)8 (6,000)Raw share: 7.27.2Bayesian estimate: 7.27.2True: 7.67.69 (8,000)9 (8,000)Raw share: 9.79.7Bayesian estimate: 9.79.7True: 9.39.310 (11,000)10 (11,000)Raw share: 8.68.6Bayesian estimate: 8.68.6True: 9.19.111 (14,000)11 (14,000)Raw share: 6.26.2Bayesian estimate: 6.26.2True: 6.26.212 (20,000)12 (20,000)Raw share: 7.87.8Bayesian estimate: 7.87.8True: 88%
  • Raw share
  • Bayesian estimate
  • True
Data as a table
MeasureRaw shareBayesian estimateTrue
1 (350)16.61414.7
2 (600)3.54.53.9
3 (900)5.86.26.1
4 (1,500)15.815.115
5 (2,200)9.59.47.9
6 (3,000)13.413.112.9
7 (4,500)1211.912
8 (6,000)7.27.27.6
9 (8,000)9.79.79.3
10 (11,000)8.68.69.1
11 (14,000)6.26.26.2
12 (20,000)7.87.88

Twelve stores: raw share, Bayesian estimate and truth. For small stores the raw share departs widely from the truth, the Bayesian estimate is closer; the mean squared error over all stores is 0.65·10⁻⁴ against 0.32·10⁻⁴. Axis: store number (visitors).

Probability of “B better than A” and the p-value as the sample grows
Line chart: P(B better than A), %: 72.2 → 99.9; p-value, %: 52.7 → 0.502040608010010020030050080012002000P(B better than A), %, 100: 72.2P(B better than A), %, 200: 81.3P(B better than A), %, 300: 86.5P(B better than A), %, 500: 91.7P(B better than A), %, 800: 96.5P(B better than A), %, 1200: 98.7P(B better than A), %, 2000: 99.9p-value, %, 100: 52.7p-value, %, 200: 37.1p-value, %, 300: 27.3p-value, %, 500: 15.7p-value, %, 800: 7.3p-value, %, 1200: 2.8p-value, %, 2000: 0.5%
  • P(B better than A), %
  • p-value, %
Data as a table
MeasureP(B better than A), %p-value, %
10072.252.7
20081.337.1
30086.527.3
50091.715.7
80096.57.3
120098.72.8
200099.90.5

Probability of “B better than A” and the p-value as the sample grows. At an observed difference of 8 → 10.6% and 500 visitors per variant the probability is 92% at p = 16%; as the sample grows at the same shares the probability tends to 100% and the p-value to zero. Axis: visitors per variant.

Estimate sensitivity to the prior at the same raw share
Bar chart: 20 9.85, 100 6.7, 500 6.01, 2500 5.8702468102020Flat Beta(1, 1): 9.85Weak (strength 10): 6.71Chain (estimated from stores): 8.18100100Flat Beta(1, 1): 6.7Weak (strength 10): 6.07Chain (estimated from stores): 7.45500500Flat Beta(1, 1): 6.01Weak (strength 10): 5.89Chain (estimated from stores): 6.4725002500Flat Beta(1, 1): 5.87Weak (strength 10): 5.84Chain (estimated from stores): 5.99%
  • Flat Beta(1, 1)
  • Weak (strength 10)
  • Chain (estimated from stores)
Data as a table
MeasureFlat Beta(1, 1)Weak (strength 10)Chain (estimated from stores)
209.856.718.18
1006.76.077.45
5006.015.896.47
25005.875.845.99

Estimate sensitivity to the prior at the same raw share. At n = 20 estimates with different priors differ by 3.1 pp, at n = 2,500 by 0.15 pp: the more data the less the prior matters. Axis: number of visitors; the raw share is 5.8%.

Width of the 95% interval for the new store
Bar chart: Chain prior 6.1, Flat prior 8.7, Wilson 8.70246810Chain priorChain priorInterval width: 6.16.1Flat priorFlat priorInterval width: 8.78.7WilsonWilsonInterval width: 8.78.7pp
Data as a table
MeasureInterval width
Chain prior6.1
Flat prior8.7
Wilson8.7

Width of the 95% interval for the new store. The informative prior gives 6.1 pp against 8.7 for the flat one and 8.7 for Wilson: the gain in precision is paid for by the assumption that the store resembles the chain.

Specifics by category

Apparel
Buyout and conversion differ across subgroups (outerwear, knitwear): the prior is built by subgroup, not for the whole category; seasonal models are estimated from analogs of the previous season.
Footwear
Returns by size and fit are high and vary widely across models: a new model has few orders, so shrinkage to the category prior is mandatory; sizes are estimated within a model.
Accessories
Few sales per SKU and many zeros: a gamma-Poisson model for the rate, a prior by group; the decision to delist an item relies on the posterior interval, not a couple of empty weeks.
Kids
A short season and size groups: little data per model, a prior by age group and last year’s season; an update after every week of sales.
Men
Demand is steadier, priors are narrower and more reliable so decisions are taken earlier; for basics shrinkage barely changes the estimate at a normal sample.
Women
Fast range turnover, little history per model and a high spread: Bayesian estimates and predictive intervals are the most useful; the prior is built by model group and color.

Common mistakes

  • A store ranking by raw shares at different n → random small samples on top → shrunk estimates and intervals next to the ranking.
  • Too strong a prior from another category → the estimate does not respond to data → a prior from the own group and a check on a weak prior.
  • The prior and its strength are hidden → the estimate cannot be reproduced or challenged → the prior version and parameters are stored with the estimate.
  • A “B better than A” probability of 92% is read as a proven difference → a launch ignoring the cost of a mistake → pre-set probability and loss thresholds.
  • A test is stopped at the first favorable probability → inflated confidence → the stopping rule and prior are fixed before launch.

Russian specifics

  • Retailer: data for a customer prior (cards, receipts with an identifier) is processed with a legal basis and in anonymized form; estimates by stores, models and days need no personal data. Rule: Law 152-FZ of 27.07.2006: Federal Law of 27.07.2006 No. 152-FZ on personal data (rule check not completed).
  • Returns and refusal of goods in distance selling are governed by the consumer protection law; return thresholds in the model are management thresholds, not norms, and do not limit buyer rights. Rule: Law 2300-1 of 07.02.1992: Law of 07.02.1992 No. 2300-1 on consumer rights protection (rule check not completed).

Open interactively →