AI in fashion and analytics: where to apply it, where not, how to check the result, which data are needed and which errors occur. A memo and a checklist
A memo on the limits of applicability of AI for a brand and a retailer: on which tasks it helps, on which it is dangerous, how to check the result against a simple rule, which data are needed and which errors occur most often. The document decides whether to introduce AI on a specific task, how to organize a pilot with a baseline and a threshold improvement and when to stop using it. It has no promises of effect: only the order of the check, an example in numbers and a checklist.
Why this document
AI looks convincing even where it is wrong: a confident text, a beautiful chart, a forecast that looks precise. So it is accepted by impression and not by a check, and the error reaches a price, a purchase or a customer. The owner, the head of data, the analyst, the planner and the head of content read the document when choosing tasks for AI and when launching a pilot. It gives a simple rule: compare with a free baseline, check on data the model has not seen and count the full cost, including preparing data and a human check.
At a glance
- What is this document and why is it needed?
- A memo on the limits of applicability of AI for a brand and a retailer: on which tasks it helps, on which it is dangerous, how to check the result against a simple rule, which data are needed and which errors occur most often. The document decides whether to introduce AI on a specific task, how to organize a pilot with a baseline and a threshold improvement and when to stop using it. It has no promises of effect: only the order of the check, an example in numbers and a checklist.
- Who owns the document and how often is it rebuilt?
- The head of data or analytics keeps the register of AI tasks and checks, the process owner answers for the result and the stop, the lawyer checks the data and rights; reviewed for every new task and once a quarter by the metrics.
- What mistakes are the most common?
- Brand: AI is accepted by the impression of a beautiful result → something weaker than a free rule is introduced → always compare with the baseline and write the improvement threshold in advance. Brand: technical descriptions, prices and factory contacts are uploaded to a public service → a leak of commercial information → use closed solutions or anonymize the data and fix the rules in a policy. Retailer: a reply to a customer from AI is sent without a check → the reply has wrong return terms or a price → check the terms and prices against reference books and keep a log of replies.
For whom
- Brand
- A brand applies AI in descriptions and images, catalog tagging, demand forecasting by SKU and customer support: it cares about the rights to images, the protection of technical documentation and purchase prices and the check of the forecast before an order at the factory.
- Retail
- A retailer applies AI in forecasting and replenishment, recommendations, replies to customers and the analysis of reviews: it cares about customers’ personal data, the liability for a reply to a buyer and the check of the replenishment forecast at the store and category level.
What it consists of
Where AI helps
- AI is useful where there are data and experience and an error is cheap or easy to see. Examples: drafts of descriptions and replies, tagging and search by product photo, extracting data from documents and waybills, grouping reviews by topic, ideas for moodboards, demand forecasting for a large number of series with a history.
- It speeds up routine but does not replace the person responsible: the result is a draft which a person checks. The gain is measured by time and errors on a specific task before and after and not by a general impression.
- Brand: use AI for drafts of product cards, tagging the catalog by photo and a first grouping of reviews, and insert prices, compositions and sizes from reference books and not from the model’s text.
- Retailer: use AI for the replenishment forecast across a large number of SKUs and stores, grouping customer requests and finding discrepancies in the data, leaving the order decision to a person with an approval threshold.
Where not to apply AI or to apply it under strict control
- Do not trust AI with calculations of money, taxes and prices without a check by a formula: it may make an arithmetic error and confidently explain a wrong result. Count by a formula in a calculation and use AI for the explanation.
- AI may invent legal and normative conclusions: a law number, a term, a fine. Check any norm in the text of the law and with a lawyer; in Lab such facts are marked “check” for exactly this reason.
- For a new SKU without a history AI builds the forecast from analogs and the quality depends on the choice of analogs: it is a hypothesis for a test and not an order. Decisions about people and personal data are not taken without a legal basis and a check.
- Brand: do not upload technical descriptions, purchase prices, contacts of factories and unpublished collections to public services: it is a trade secret; use closed solutions or anonymize the data.
- Retailer: a reply to a customer generated by AI is a reply of the company: check the return terms, the dates and the prices before sending and do not hand customers’ personal data to a service without a basis and a contract.
How to check the result
- The baseline: compare AI with a simple free rule (as in the last period, as in the same period of the last season). A method that does not beat the baseline is not needed however it looks. The gain is counted as one minus the AI error divided by the baseline error.
- Holdout data: check on periods that the model did not see during tuning and fix the forecast before the fact. Fitting to a known fact gives a beautiful and useless figure. Look not only at the average error but also at the bias (overstates or understates) and at the share of periods where the AI is more accurate.
- For texts and images: a gold set of answers known in advance, a manual sample (a check of a given number of results by a person), repeatability (are the conclusions the same for the same query) and sensitivity to wording. Check any fact in a text against the data source.
- Brand: before an order at the factory by an AI forecast compare it with the naive rule on the last season and write down the improvement threshold at which the forecast is used; below the threshold the simple rule remains.
- Retailer: check the replenishment forecast at the level of a store and a category separately: a forecast accurate on average across the chain may be poor for individual stores and sizes.
Data and typical errors
- Sales are not demand: when goods are short or an outlet is closed the demand is not seen. Duplicates, gaps, changes of SKU codes, promotions and one-off orders distort a series; before training clean the data and mark the periods of shortage and promotions.
- Typical errors: leakage of the future (data that did not exist at the time of the forecast got into training), fitting to history, drift (customers’ behavior changed and the model did not), a biased sample, invented facts in texts and excessive confidence. Against each there is a check: holdout data, monitoring the error by weeks, a manual sample.
- Brand: keep a register of AI tasks: what it does, on which data, who answers, which metric and stopping threshold; without a register nobody knows what works and how much it costs.
- Retailer: monitor the forecast error by week and by groups of stores and stop using it when the error goes beyond the threshold; return to the simple rule until the cause is found.
The pilot checklist
- Before launch: the task and the decision it supports are named; the data and their owner exist; the baseline is defined; the improvement threshold is set; the cost of an error is estimated; the rights to the data and images are checked; the person responsible and the stopping threshold are defined.
- In operation: the result is compared with the baseline on holdout data; a manual sample is kept; the error and bias are tracked by week; the log of queries and versions is kept; personal and commercial data do not go to public services.
- After: the gain is counted in money net of the full cost (the subscription, the preparation of data, the human check); the decision: widen, keep, stop; the conclusions are recorded in the register.
Where the data comes from
| Data source | Fields |
|---|---|
| Historical sales and stock | sales by SKU and period, marks of shortage and promotions, code changes, prices and the calendar |
| The forecast and the model versions | the forecast made before the period, the date and the version, the parameters, the horizon |
| The baseline and the metrics | the naive forecast, WAPE, bias, the share of periods, the improvement threshold |
| The usage log | queries and replies, the manual sample and remarks, the cost, the person responsible, the stopping decisions |
How to build it step by step
- Step 1: Brand: choose a task, name the decision it supports and the cost of an error; do not take tasks without a decision for AI.
- Step 2: Check the suitability matrix: are there data and experience, is an error cheap, can the result be checked.
- Step 3: Prepare the data: clean duplicates and gaps, mark shortage and promotions, check the rights and personal data.
- Step 4: Define the baseline and the improvement threshold before the pilot starts and write them down.
- Step 5: Make a forecast before the fact and compare it with the baseline on holdout periods: WAPE, bias, the share of periods.
- Step 6: Retailer: check the result at the level of a store and a category separately and add a manual sample of replies if they are texts.
- Step 7: Count the gain in money net of the full cost and compare with the threshold; decide: widen, keep or stop.
- Step 8: Retailer: set up the monitoring of the error by week and a stopping threshold, appoint a person responsible.
- Step 9: Brand: enter the task in the register with the metric, the cost and the date of the next check.
Formulas
| Measure | Formulas | Note |
|---|---|---|
| WAPE and bias | WAPE and bias are defined once: in the term “Forecast error (WAPE) and bias” and in the “Forecast accuracy by week” calculation (fcdash); here they are counted over the same periods for the AI and for the naive rule | A plus in the bias overstates sales, a minus understates. |
| Improvement against the baseline | Improvement = 1 − AI WAPE ÷ naive WAPE | Compared with the threshold set by policy. |
| Gain | Gain = (naive error − AI error) × cost of an error of a unit − cost of the tool | The costs include preparing data and a human check. |
Formulas in notation
- Improvement and gain
I = 1 − WAPE_ai / WAPE_naive, G = (E_naive − E_ai) · c − K
E: the total absolute error in units; c: the cost of an error of one unit; K: the cost of the tool over the horizon; the method is adopted if I is not below the threshold and G > 0.
Worked example
An illustrative check of the demand forecast for the shirt-jacket: 12 weeks, AI against the rule “as last week”
| Indicator | AI | Naive rule |
|---|---|---|
| Weeks compared | 11 | 11 |
| Total error, units | 181 | 215 |
| WAPE, % | 12.1 | 14.4 |
| Bias, % | 0.3 | −0.3 |
| Weeks where the method is more accurate | 9 | 2 |
| Improvement against the rule, % | 15.8 | — |
| Gain (34 units × RUB 600) and the result after RUB 20,000 of cost, RUB | 20,400 / 400 | — |
Illustrative data. The improvement of 16% passes the threshold of 10% but the gain almost equals the costs: the result is about RUB 400 over 12 weeks. It is a reason not to widen the use but to check one more season and the cost of preparing the data. A tie in one week counts for the rule.
Figures
- Actual
- AI forecast
- Naive rule
Data as a table
| Measure | Actual | AI forecast | Naive rule |
|---|---|---|---|
| W2 | 95 | 85 | 80 |
| W3 | 110 | 98 | 95 |
| W4 | 140 | 116 | 110 |
| W5 | 170 | 146 | 140 |
| W6 | 190 | 172 | 170 |
| W7 | 185 | 201 | 190 |
| W8 | 160 | 178 | 185 |
| W9 | 140 | 156 | 160 |
| W10 | 120 | 134 | 140 |
| W11 | 100 | 114 | 120 |
| W12 | 85 | 100 | 100 |
Actual, the AI forecast and the naive rule by week. The naive rule lags on the rise and on the fall of demand; the AI errs less but overstates sales at the peak.
Data as a table
| Measure | WAPE, % |
|---|---|
| AI | 12.1 |
| Naive rule | 14.4 |
WAPE error: AI and the naive rule. The AI is more accurate by 2.3 pp, an improvement of 16% against the rule; the question is whether it pays off the costs.
- 0 … 1 0 or 1
Data as a table
| Data and experience exist | An error is cheap | The result is checkable | |
|---|---|---|---|
| A draft product description | 1 | 1 | 0 |
| Tagging and search by product photo | 1 | 1 | 1 |
| Demand forecast for SKUs with a history | 1 | 0 | 1 |
| Forecast for a new SKU without a history | 0 | 0 | 0 |
| A legal or tax conclusion | 0 | 0 | 0 |
| A draft reply to a customer | 1 | 1 | 0 |
| Automatic price selection without economics | 1 | 0 | 0 |
Suitability of tasks for AI: an illustrative score on three questions. 1 means “yes”, 0 “no” on each question; the more ones, the lower the risk. The score is a judgment and not a measurement: put in your own answers.
An AI pilot: from a task to a decision. The baseline and the threshold are set before the pilot; the decision is made by holdout data and cost.
- Task and decision → Data and rights
- Data and rights → Baseline and threshold
- Baseline and threshold → Test on holdout data
- Test on holdout data → Gain net of costs
- Gain net of costs → Widen, keep, stop
- Widen, keep, stop
Specifics by category
- Apparel
- Apparel: AI is useful in tagging the catalog by photo and in forecasting by SKUs with a history; forecast new items without a history by analogs as a hypothesis and check the size grid separately because an error by sizes gives both a shortage and stock.
- Footwear
- Footwear: the main error is sizes and lasts: AI-based size recommendations reduce returns only when checked on returns data; check the forecast by size run for every size and not on average.
- Accessories
- Accessories: there are few sales per SKU, a forecast by SKU is noisy; forecast at the level of a category and a type and apply AI for tagging, finding similar items and descriptions where an error is cheap.
- Kids
- Children’s: data about children and their parents are personal data with a special regime and are not uploaded to AI services without a basis and a check; demand is strongly seasonal and tied to size, so a naive rule with a seasonal shift is often hard to beat.
- Men
- Men’s: demand is more stable and repeats for basic items, so a simple rule with a seasonal shift is strong; AI wins on the long tail of SKUs and in tagging and not on the core where the gain over the baseline is small.
- Women
- Women’s: demand is more volatile, there are many trend items and short life cycles, the history is shorter; the forecast for new SKUs remains a hypothesis and the gain of AI is more often noticeable in customer behavior and content than in the accuracy of the forecast of individual SKUs.
Common mistakes
- Brand: AI is accepted by the impression of a beautiful result → something weaker than a free rule is introduced → always compare with the baseline and write the improvement threshold in advance.
- Brand: technical descriptions, prices and factory contacts are uploaded to a public service → a leak of commercial information → use closed solutions or anonymize the data and fix the rules in a policy.
- Retailer: a reply to a customer from AI is sent without a check → the reply has wrong return terms or a price → check the terms and prices against reference books and keep a log of replies.
- Retailer: the forecast is checked only on average across the chain → it is poor for individual stores and sizes → assess by stores, categories and sizes separately and watch the bias.
- The check is run on the data the model was tuned on → the error is understated and in operation it is higher → keep the periods for checking and fix the forecast before the fact.
Russian specifics
- Federal Law of 27.07.2006 No. 152-FZ on personal data: uploading customers’ data to an AI service is processing, often an instruction to a contractor and possibly a cross-border transfer; check the basis, consent and localization with a lawyer against the current edition. Rule: Law 152-FZ of 27.07.2006: Federal Law of 27.07.2006 No. 152-FZ on personal data.
- Brand: the rights to images and texts created with AI, the use of other people’s images, faces and style, and also the requirements for disclosing the use of AI in advertising and content are governed by the current legislation and the terms of the service: check with a lawyer, unverified.
- Retailer: a reply to a customer prepared by AI is a reply of the company: the return terms, the warranties and the information about the goods must match the consumer protection law and your rules; check the wording with a lawyer, unverified.
- Brand: a trade secret (purchase prices, factories, unpublished collections) is protected by the trade secret regime and contracts; uploading to public services may break the regime: check the procedure with a lawyer, unverified.