Silicon fab AI benchmarks: making a wafer-fab AI number comparable

Rung 1 is not ignorance. Fabs at this rung are usually extremely good at measuring silicon: they run SPC on hundreds of parameters, they hold e-test and parametric data for years, and they can tell you a chamber's particle count by shift. What they do not have is a *measurement contract for change* — a stated way of establishing what a metric was before an intervention, so that what it becomes afterwards means something.

The tell is the provenance of the numbers in the AI business case. Ask where 30% came from and the honest answer is a slide. That slide is not dishonest; it describes a real result at a real fab with a particular product mix, a particular tool generation and a particular sampling plan, none of which are printed on the slide. Transplanted into a different fab it is not an estimate of anything — it is a number-shaped object.

This rung is cheap to leave and expensive to occupy, because borrowed numbers set expectations the fab then has to meet without any means of showing whether it has. The first real deliverable is not a model. It is a frozen extract of six to twelve months of MES lot history, equipment traces and metrology results for one process area, stored immutably, with the product mix and node labelled on every lot.

Rung 2 is the most dangerous rung on the ladder, because it produces numbers that look exactly like benchmarks. The model went in; the metric improved; the improvement was calculated from the fab's own data. Everything about that sequence is defensible except the inference, because in the same window the mix shifted toward a higher-yielding product, two chambers came out of preventive maintenance, and the metrology sampling plan was relaxed on a mature layer.

The structural problem is that a wafer fab is a poor natural experiment. Its output is driven by more large, slow-moving confounders than almost any other manufacturing environment: technology node, product mix, tool generation, reticle set, ramp phase, seasonality in the customer order book, and the yield-learning curve itself, which improves a mature line steadily whether or not anyone deploys anything. A before-and-after comparison in that environment attributes all of it to the last thing that changed.

Time at rung 2 is not neutral. The first time a self-reported gain is challenged — usually by finance during a capital review, or by a customer quality auditor asking how an AI-influenced disposition decision was validated — the number collapses, and the programme spends its credibility rebuilding what it should have designed at the start. Fabs that sit at rung 2 for three years are typically harder to move than fabs at rung 1, because the organisation has learned to discount AI numbers on sight.

Rung 3 is where a fab acquires an internal currency. Cycle time means one thing; availability means one thing; a wafer counts once. That sounds trivial and it is the single most contested piece of work on the ladder, because a shared definition necessarily overrides several local ones, and each local definition exists because it was useful to somebody. The standards help enormously here — SEMI E10 for equipment states, E79 for OEE, E116 for performance tracking — precisely because they let the argument be settled by reference rather than by seniority.

The second half of rung 3 is normalisation, and it is the part fabs consistently underestimate. A benchmark that has not been stratified by product family, node and tool generation is measuring the order book. The published virtual-metrology literature makes the same point from the other direction: an uncertainty-aware study of phototransistor gain found that roughly half of the variance sat *between* process runs rather than within them, which bounds by construction how much a recipe-only model can ever predict. Knowing where variance lives is what tells you which normaliser you need.

The constraint that emerges at rung 3 is causal rather than statistical. Numbers are now reproducible and stratified, so the fab can honestly say what changed — but not yet why. Everything is still a before-and-after on the same tools, and the yield-learning curve keeps improving the line underneath the measurement. The next rung costs discipline rather than money: leave part of the fleet on the old method on purpose.

Rung 4 is where measurement stops being reporting and becomes experiment design. The mechanics are old and well documented — the NIST/SEMATECH e-Handbook has carried them for decades — but they are unfamiliar to most fab AI programmes because pilots are usually run as deployments rather than as trials. A split-lot design randomises lots between the AI-supported and existing paths across chambers and shifts; a holdout-chamber design keeps a qualified chamber on the incumbent recipe control. Either gives the fab a contemporaneous control, which is the only thing that removes the learning curve from the answer.

The second discipline is power. Fab effect sizes are small — a fraction of a yield point, a few percent of X-factor — and lot counts are limited, so a design that has not been powered in advance will usually fail to detect an effect it cannot afford to miss, then be reported as 'no significant difference'. Compute the required lot count before the trial and, if it is unaffordable, say so and choose a different metric rather than running an underpowered trial and reading the result anyway.

The third is disclosure, and it is what makes the number portable. A benchmark travels with its panel: which toolsets, which product families, which window, how many lots, what was held constant. The published wafer-fab scheduling case studies that practitioners actually quote are the ones that state this — a reported 24% average cycle-time reduction across 11 toolsets is quotable precisely because the panel is on the page. A number without a panel cannot be reused, cannot be challenged and cannot be reproduced.

Rung 5 is narrower than it sounds, and deliberately so. It does not mean the fab benchmarks everything; it means a defined, enumerated set of claims — the ones attached to decisions that touch product disposition, sampling reduction or dispatch — is recomputed on a schedule and stands between a model and production. Everything outside that set is still measured, still reported and explicitly not treated as benchmark-grade, which is a distinction worth writing down.

The engineering is mostly done by the time a fab arrives here. What is hard is the governance artefact: the benchmark specification itself, versioned, with a change history that records when the panel changed and why. Panels decay. A product family goes end-of-life, a toolset is upgraded, a metrology recipe changes, and a benchmark computed on the old panel silently becomes a comparison between two different fabs. The change log is what lets an auditor — or your own team in two years — tell a real improvement from a panel edit.

This is also the rung at which benchmarks stop being an internal convenience and start being external evidence. Customer quality audits ask how an AI-influenced disposition or sampling decision was validated; process-change notification under the JEDEC framework asks the supplier to describe and justify a change. A governed benchmark set answers both as an export rather than as a project, which is usually the argument that finally funds the work.