Syntha Lab RU

Data quality and data owners

Completeness, uniqueness, validity and timeliness of reference books, an overall score and the cost of errors; who is accountable for each reference book (RACI).

both Processes and data

Open interactively →

About the topic

Data quality is not a feeling that things are fine but a set of measurable properties of a reference: how complete it is, how free of duplicates, how correct in format, how fresh and how consistent between systems. Until quality is measured it is discussed after an incident: an order went in the wrong size, labeling failed, a report did not reconcile with the ledger. After measurement you have a number, a weak spot and an owner.

Why it matters to a brand or retailer. Bad data is expensive unnoticeably: manual reconciliation, rework of cards, rejections at receiving, document returns, selling non-existent stock. The decisions on this topic: which reference to start with, which dimension to fix first, how much to spend on cleaning, who answers for each reference and how often to check.

The logic is built on dimensions. Completeness asks whether all mandatory fields have values. Uniqueness: are there no duplicates and one code for two products. Validity: do values pass a format check (barcode check digit, tax-number length, an allowed color from the list). Timeliness: was the record updated within the required time. There are also accuracy (does the record match reality) and consistency (the same record in different systems says the same). The first four are counted from reference counters and are easily automated; the last two need sample checks against the world and between systems.

A measurement becomes a score through weights: you decide what matters more. But an overall score is dangerous, because an average hides a failure in one place. A reference may score 92% while completeness of 85% means almost every seventh card is not ready for publication or labeling. So next to the overall score keep thresholds for each dimension and stop conditions: a card with a duplicate barcode does not go into the exchange, however high the overall score. The second half of the topic is accountability. A reference must have one accountable person (A in the RACI matrix) who decides what counts as the norm, and an executor (R) who fixes. If accountability is a little with everyone, nobody answers.

Where to start: take the SKU reference, set mandatory attributes and update term, compute counters, find the weakest dimension and estimate the price of errors from the incident log. Repeat each period and keep a log of rule changes, otherwise trends cannot be compared. Traps: measuring quality once before launch, fixing errors in the report instead of the source, adding error causes without accounting for overlaps, changing check rules without a record.

Key ideas

Four dimensions are counted from counters

12,000 SKUs: all mandatory fields filled for 10,200 (completeness 85%), a duplicate barcode on 120 (uniqueness 99%), a wrong or empty barcode on 60 (validity 99.5%), not updated on time 900 (timeliness 92.5%). Everything is divided by total SKUs, so the dimensions are comparable.

The overall score depends on weights and hides the weak spot

Weights 40, 20, 20, 20 give 0.4 × 85 + 0.2 × 99 + 0.2 × 99.5 + 0.2 × 92.5 = 92.2%. The result looks fine though 1,800 cards are incomplete. If duplicates grow to 1,200 (uniqueness 90%), the score falls only to 90.4%: the weight is too small to notice the problem. Critical dimensions need their own thresholds and stop conditions.

The price of errors is estimated as a range

1,800 incomplete cards. The incident log shows that 4–12% of them cause a failure (rejection at receiving, return, rework), and each failure costs 2,500 roubles. Lower estimate 1,800 × 0.04 × 2,500 = 180,000 roubles, upper 1,800 × 0.12 × 2,500 = 540,000 roubles. A range is more honest than one number: it shows that cleaning is justified if it costs less than the lower bound.

One accountable, many executors (RACI)

The SKU reference: accountable (A) the category head, executor (R) the data specialist, consulted (C) logistics and marketing, informed (I) finance. A is one person: he or she decides what counts as the norm. If column A has three names, the rule everyone a little means the decision is postponed.

Fix the source, not the report

An analyst manually fixes the color blk in the export every week. That is an hour a week, 52 hours a year, and the error returns with the next export. A fix in the reference is made once and disappears from all reports. For the check to work, the rules and their changes are recorded in a log.

Stock movement and sales are reconciled by an invariant

The expected stock is 10,650 pcs against the actual 10,400: a gap of −250 pcs. The till and the ledger differ by RUB 50,000, the bank differs from the expectation by −RUB 60,000: the effective fee is 2.86%.

Formula: Expected end = beginning + receipts − sales + returns − transfers − write-offs + adjustments

What to do: Reconcile stocks at one date.

How to work through it

  1. Choose a reference to check: usually SKU cards, because everything else depends on them.
  2. Set the mandatory attributes, allowed formats and the update term of each field.
  3. Compute counters: filled, duplicates, wrong codes, overdue records.
  4. Get the weighted score and separately show the weakest dimension and stop conditions.
  5. Estimate the price of errors from the incident log as a range.
  6. Assign accountable A and executor R by RACI; agree who reads the result.
  7. Fix the source, not the report, and keep a log of check-rule changes.
  8. Repeat the check every period and compare the trend with the past.

Where to start

1) Choose a reference book to check: SKU cards. 2) Set the mandatory attributes and the update period. 3) Count the counters: filled, duplicates, invalid codes, stale. 4) Get the score and the weakest dimension. 5) Estimate the price of errors from the incident log. 6) Assign the accountable A and the doer R by the RACI matrix. 7) Repeat the check every period and keep a change log of the rules. Calculations of the topic in order: “Completeness and quality of product master data”, “Stock roll-forward reconciliation: beginning + receipts − outflows = end”, “Sales reconciliation: till, ledger, acquiring and payout”. See also (their primary topic is another one): “Stock record accuracy and the cost of mismatches”, “Stock sync latency: the price of stale data”.

Calculations in this topic

  • Completeness and quality of product master data

    From the number of SKUs, filled cards, duplicates, invalid barcodes and stale records calculates four quality dimensions, an overall score by your weights and the cost of errors as a range.

    How ready the SKU reference book is for work and what errors cost.

  • Stock roll-forward reconciliation: beginning + receipts − outflows = end

    Checks the stock movement invariant: the beginning stock plus receipts, minus sales, plus returns, minus transfers and write-offs, plus adjustments must give the ending stock. Counts the gap against the actual in units, in percent of turnover and in rubles, compares it with the tolerance and hints at which line and sign to look at first.

    Whether the stock movement adds up and where to look if not.

  • Sales reconciliation: till, ledger, acquiring and payout

    Reconciles the net sales at the till (gross minus returns and discounts) with the amount in the ledger, and the expected non-cash receipt (non-cash sales minus the acquiring fee) with the real receipts on the account. It accounts for the settlement lag, shows the gaps, compares with the tolerance and counts the effective fee that explains the difference with the bank.

    Whether the till, the ledger and acquiring money match.

  • Stock record accuracy and the cost of mismatches

    From a sample count it calculates record accuracy by lines and by value, the net balance of mismatches and estimates the lost revenue from “phantom” stock: an item is on record but not on the shelf.

    How far stock records match reality and what the mismatches cost.

  • Stock sync latency: the price of stale data

    From the number of orders, the share of orders on low-stock SKUs and the sync interval calculates how many orders a month sell what is already gone, what it costs and whether more frequent refresh pays back.

    What stale stock on a channel costs and whether more frequent refresh pays back.

Self-check questions

  • What are completeness, uniqueness, validity and timeliness of the SKU reference?
  • Which dimension is weakest and where should cleaning start?
  • Which stop conditions do not let a card through regardless of the overall score?
  • What do errors cost by the incident log in the lower and upper estimate?
  • Who is accountable for each reference and who is the executor?
  • How often do you check the reference and who reads the result?
  • Does the same record match across systems (consistency) and how is it checked?
  • Do you record changes to check rules so periods can be compared?

Knowledge check

Multiple-choice questions. The correct answer and the explanation follow each question.

  1. For 30 SKUs the barcode is filled but fails the check-digit test. Which data-quality dimension suffers?

    • A. Completeness
    • B. Uniqueness
    • C. Validity
    • D. Timeliness

    Correct answer: C. Validity. The value exists but is invalid by format: that is validity (1 − invalid ÷ all). Completeness does not suffer, the field is filled. Uniqueness is the absence of duplicates, and there is no mention of them here; timeliness is about how recently a record was updated, not about the code being right.

  2. The reference book has 5,000 SKUs: 4,250 are fully filled, 50 have a duplicated barcode, 25 an invalid code, 400 are not updated on time. The weights of completeness, uniqueness, validity and timeliness are 40, 20, 20 and 20. What is the overall score?

    • A. 92.1%
    • B. 93.9%
    • C. 85%
    • D. 96.8%

    Correct answer: A. 92.1%. Completeness 4,250 / 5,000 = 85%, uniqueness 99%, validity 99.5%, timeliness 92%. Score = (40 × 85 + 20 × 99 + 20 × 99.5 + 20 × 92) / 100 = 92.1%. 93.9% is the simple average without weights, 85% is the weakest dimension alone (completeness), and 96.8% comes from dropping completeness: the weakest dimension and the heaviest by weight.

  3. A reference book of 2,000 SKUs: 300 incomplete cards, 40 duplicated codes, 20 invalid codes, 200 stale records. 5% of problem SKUs cause an incident, one incident costs RUB 2,000. In what range is the cost of errors per period?

    • A. exactly RUB 56,000
    • B. exactly RUB 30,000
    • C. from RUB 600,000 to RUB 1,120,000
    • D. from RUB 30,000 to RUB 56,000

    Correct answer: D. from RUB 30,000 to RUB 56,000. The causes overlap, so a range is calculated: the lower bound is the biggest single cause (300 SKUs), the upper is the sum of causes (300 + 40 + 20 + 200 = 560 SKUs). 5% cause an incident: 300 × 0.05 × 2,000 = RUB 30,000 and 560 × 0.05 × 2,000 = RUB 56,000. A single figure, 56,000 or 30,000, would be false precision; RUB 600,000–1,120,000 comes from forgetting the share of SKUs that caused an incident.

  4. In the RACI matrix of the supplier reference book there are two As (buying and finance) and no R. What should be fixed?

    • A. Keep two accountable parties: the responsibility is split in half
    • B. Keep one accountable A, move the others to C or I and assign at least one doer R
    • C. Replace both As with R: let everyone do it
    • D. Make the whole buying department accountable

    Correct answer: B. Keep one accountable A, move the others to C or I and assign at least one doer R. A RACI row has exactly one A and at least one R. Two As mean that in a dispute “everyone a little” is responsible, that is, nobody; without an R nobody corrects the records; if everyone is R there is no accountable party; a department instead of a person is not accountable either: a specific person is assigned.

  5. A sample count: 250 lines checked, 230 accurate. Surpluses of RUB 150,000, shortages of RUB 250,000, book value of the checked lines RUB 10,000,000. What are the accuracy by lines and the value error?

    • A. 92% and 4%
    • B. 92% and 1%
    • C. 8% and 4%
    • D. 92% and 2.5%

    Correct answer: A. 92% and 4%. Accuracy by lines = 230 / 250 = 92%. Value error = (surpluses + shortages) / book value = 400,000 / 10,000,000 = 4%. 1% comes from taking the net balance (250,000 − 150,000): surpluses and shortages cancel each other, but the record error stays. 8% is the share of inaccurate lines, and 2.5% is the shortages alone.

  6. The actual is below the record. Where to look first?

    • A. Unposted receipts
    • B. The price
    • C. Transfers and write-offs, short receipt
    • D. The currency rate

    Correct answer: C. Transfers and write-offs, short receipt. Actual below: an unposted outflow. Covered in: Stock movement and sales are reconciled by an invariant.

  7. Why does the bank differ from the expected receipt?

    • A. The fee, holds, card refunds and the settlement lag
    • B. Only a cashier mistake
    • C. The currency rate
    • D. Seasonality

    Correct answer: A. The fee, holds, card refunds and the settlement lag. The effective fee shows the difference. Covered in: Stock movement and sales are reconciled by an invariant.

Common mistakes

  • Mistake: Measuring quality once before launch

    What it costs: Nobody notices deterioration

    How to avoid: A check every period with a trend.

  • Mistake: Naming everyone a little as accountable

    What it costs: Decisions are postponed

    How to avoid: One A per reference.

  • Mistake: Adding error causes without accounting for overlaps

    What it costs: The price of errors is overstated

    How to avoid: Count unique cards with an error.

  • Mistake: Changing check rules without a log

    What it costs: Trend cannot be compared

    How to avoid: Record the date and reason of every change.

  • Mistake: Fixing errors by hand in the report

    What it costs: They return with the next export

    How to avoid: Fix the source.

  • Comparing stocks of different dates

    What it looks like (illustrative figures): The gap is spurious.

Russia and the EAEU

There is no unified Russian standard on data quality: you set thresholds and update terms yourself. But some requirements turn quality into an obligation. If references store personal data of customers, Federal Law 152-FZ applies: data must be accurate and up to date, and the subject may demand clarification; data of Russian citizens is stored in databases in Russia (check with a lawyer). For labeled goods a wrong GTIN or an empty attribute in the National Catalog leads to rejection when codes are put into circulation. In electronic documents (UPD, UKD) errors in requisites and names require correction, and an error in a VAT invoice that prevents identifying the seller, the goods or the tax rate may put the VAT deduction in question.

Format checks can be automated: a legal entity’s tax number has 10 digits, an individual’s 12, and both contain check digits; a GTIN barcode is checked by its check digit; a customs nomenclature code has ten characters. Checking a counterparty against a register before the first deal reduces the risk of errors in requisites.

The quality dimensions are the same; identifier formats and languages change. In Belarus the taxpayer identifier is the UNP, in Kazakhstan the BIN for organizations and IIN for individuals, in Kyrgyzstan the INN; lengths and check digits differ, so format checks must be set by country, not by the Russian pattern (reference, check). Names and addresses may be in two languages, and the reference must store both rather than mix them in one field. For deliveries to Russia the product card must carry the union’s customs nomenclature code and, separately, national catalog attributes. Requirements for storing and protecting personal data differ by country; check with a lawyer before connecting a shared CRM.

Categories and audiences

Apparel
In apparel the main sources of error are size and color (text instead of a list), composition and care. Completeness of composition is critical for labeling and documents, so it is kept as a separate threshold. Make sure one size does not live in two spellings.
Footwear
In footwear the weak spots are the size scale (EU, RU, Mondopoint), completeness of last and materials and one pair matching one code. A duplicate barcode on two sizes breaks the warehouse. Check size validity against the scale reference, not the text.
Accessories
Accessories are fewer, so quality is often higher by counters, but errors cost more: sets, contents, materials and customs codes are varied. Check that a set has its own code and a link to its content, and that a goods item without size has no artificial size.
Kids
In children’s goods age, height and certificate information are critical: permission to sell depends on them. An empty age or certificate field costs more than in other categories, so a stop condition is set for it. Pre-school seasonal updates require fresh data.
Men
In the men’s master data errors accumulate on links: a carried-over model lost its reference to the predecessor when the SKU changed, and the size pair is written as “50-182” and “50/182”. So measure the share of carried-over models with a filled predecessor and the uniformity of the pair format: year-on-year comparability of seasons depends on them.
Women
In the women’s line color and size are the weak spot: “beige” written three different ways is counted as three colors. Measure the share of cards where the color comes from the reference list and sizes from the approved grid. During a collection launch check it weekly, until cards lag behind prices and promotions.

Questions this topic answers

  • What are the completeness, uniqueness, validity and timeliness of the SKU reference book?
  • Which dimension is the weakest and where to start the clean-up?
  • What do data errors cost by the incident log?
  • Who is accountable for each reference book and who is the doer?
  • How often to check the reference book and who reviews the result?

Key measures and terms

Study first

What next

Next: “KPI dictionary and common definitions”, “Data model of standard reports”, “Stock accuracy, shrink and availability”.

Documents on this topic

Open interactively →