Insights

Scoping an ML pilot so procurement can compare vendors

Buyers compare data and ML vendors on slide decks. Comparable pilots compare a question, a time box, and evidence you can defend.

Most “PoC” requests fail the same way: each supplier picks a different dataset, a different success story, and a different clock. Procurement then has three demos and no shared score. The fix is a one-page scope that every bidder must accept before they start.

Write the question first

State the decision the business will make if the work succeeds. “Can we forecast weekly demand well enough to cut emergency purchases by 15% in two distribution centers?” is a question. “Show us your platform” is not. If two vendors cannot answer the same question, you are not running a pilot — you are collecting theater.

Fix the time box and the data

Give a calendar window (four to eight weeks is enough for a first look) and a frozen extract: tables, grain, known quality issues, and what must not leave the building. Do not let a vendor invent a cleaner sample. The point is how they work with the mess you actually have.

Name the evidence, not the stack

Ask for: the metric you will judge, a baseline (last year’s rule, last year’s forecast, a simple model), an error analysis on the cases that matter to operators, and a handoff note a maintenance team can read. Tools are optional. If a proposal leads with licenses and buries the baseline, treat that as a red flag.

Make the comparison cheap

Use the same scorecard for every bidder (see the procurement brief). Score capability, delivery, and how they talk about risk — including how they would describe a failure to your audit committee. Then pick one vendor for a paid next step, or none. A fair “no” is a successful process.

If you want a template, download the brief and email us the question you are trying to answer.

Email us

Download the brief (PDF) (PDF in English)