Redefining Technology

Energy & UtilitiesAI Implementation & Best Practices

AI operator assist in energy and utilities control rooms: from alarm noise to supervised autonomy

AI operator assist is decision support that works inside a live electricity control room — grouping an alarm flood into one event, ranking probable causes, drafting the next best action during switching and restoration. It does not replace the certified operator who authorises the action. It changes how fast that operator can see, understand and decide.

Illustrative scene: control-room operators working in front of a wall of network, alarm and trend displays
Energy & Utilities · AI Implementation & Best Practices

Key takeaways

  1. Operator assist is an HMI problem before it is a model problem. A suggestion that appears on a second monitor is a suggestion that is not read during the ten minutes it matters, because the operator's eyes are on the alarm summary and the single-line diagram they are accountable for.
  2. You cannot assist an unrationalised alarm system. EEMUA 191, quoted by the UK Health and Safety Executive, sets a long-term average of no more than one alarm every ten minutes per operator in normal operation and no more than ten in the first ten minutes after a major upset; below that discipline, AI simply summarises noise faster.
  3. Trust fails in both directions and both destroy value. Over-trust turns a certified operator into a button-presser who stops verifying; under-trust turns a funded programme into an ignored panel. The design variable that separates them is how cheaply the operator can check the suggestion on the screen they are already looking at.
  4. The ladder runs Alarm noise → Rationalised → Assisted → Guided → Supervised autonomy, and rungs two and three are where the operational value actually appears. Rung five is a small, enumerated set of bounded actions — not an unattended control room.
  5. Measure the control room, never the operator. Desk-level and event-level measures — alarms presented per flood, time to first correct action, suggestion acceptance and override reasons, customer minutes lost on covered feeders — answer the funding question without creating a surveillance instrument that certified professionals will rightly refuse.

Abbreviations used on this page

SCADA
Supervisory control and data acquisition — the telemetry and control layer
EMS
Energy management system (the transmission control room's master application)
ADMS
Advanced distribution management system (the distribution control room's equivalent)
OMS
Outage management system — the incident and restoration record
HMI
Human-machine interface — the console the operator actually works in
SLD
Single-line diagram, the network mimic on the operator's screen
RTU
Remote terminal unit — the substation device that reports to SCADA
EEMUA
Engineering Equipment and Materials Users Association, publisher of EEMUA 191 on alarm systems
ISA
International Society of Automation, publisher of ANSI/ISA-18.2 alarm management
NERC
North American Electric Reliability Corporation — certifies system operators and sets reliability standards
FLISR
Fault location, isolation and service restoration
CML / CI
Customer minutes lost and customer interruptions — the GB distribution reliability measures

Free · 8 questions · ~3 minutes

Score your control room on the assist ladder

Eight questions, one at a time, about three minutes. Answer them and we build your personalised report — your rung on the ladder, your score on each of the four dimensions, and the specific blocker between your desk and the next rung — and send it to your inbox. Every question is about the desk and the console, never about an individual operator.

0 of 8 answered

Question 1 of 8Alarm and signal quality

What is the steady-state alarm rate per operator on your busiest desk in normal operation?

EEMUA 191, quoted by the HSE, sets a long-term average of no more than one alarm every ten minutes per operator. Above that band, interpretation work is being pushed onto a channel that is already full.

How the score maps to a stage
  • 05 — Stage 1, Alarm noise. The console is saturated: alarms without defined responses, standing and chattering points, and floods that the operator survives on experience alone.
  • 611 — Stage 2, Rationalised. Every alarm has a defined operator response, rates are measured against a published benchmark, and floods are characterised rather than endured.
  • 1216 — Stage 3, Assisted. AI adds interpretation inside the operator's own screens — flood grouping, cause candidates, 'what changed' summaries — while every decision stays exactly where it was.
  • 1721 — Stage 4, Guided. The assist proposes a next best action — a switching sequence, a restoration order, a relief action — checked against constraints, and the operator authorises, edits or rejects it.
  • 2224 — Stage 5, Supervised autonomy. A small, enumerated set of bounded actions executes without per-action approval inside a versioned envelope, with the operator supervising, inhibiting and handling everything outside it.

What AI operator assist is in an electricity control room

A definition, the line it must not cross, and the path an alarm actually travels from a fault to an authorised action.

AI operator assist is decision support that runs inside a live electricity control room and works on the operator's own screens: it groups an alarm flood into a single event, ranks probable causes with their evidence, projects when a limit will be reached, and — higher up the ladder — drafts the switching or restoration sequence for a human to authorise. It is not autonomy, and it is not a dashboard. It is a change to what a certified operator sees and how quickly they can act on it.

The line it must not cross is defined by accountability rather than by technology. A control-room operator is a certified professional carrying personal responsibility for the network: in North America through NERC's System Operator Certification (opens in a new tab) credentials, in Great Britain through the authorised-person regime under the safety rules, and everywhere through the plain fact that their name is on the switching programme. Assist may change what that person perceives, comprehends and can project. It may not quietly become the thing that decides, because there is no mechanism by which accountability follows the suggestion.

That constraint is what makes this an implementation problem rather than a modelling one. The interesting questions are where a suggestion renders, how fast its evidence can be reached, what it does when its inputs are bad, how an override is captured, and how you prove any of it worked without building a surveillance instrument. Where the suggestion is computed — an optimiser, a classifier, a physics model — matters far less than the six inches of screen it has to arrive on. If you want the optimisation methods themselves, they are the subject of the sibling page on AI grid and layout optimisation; this page starts where that page's answer meets a human being.

How an alarm becomes an authorised action, by rung

The same fault, three control rooms. The rung is set by where the interpretation happens: in the operator's head from raw rows, in the console alongside the rows, or inside a bounded envelope that acts and then tells the desk what it did. Most control rooms are in the top lane.

  • Data & feeds
  • Where value leaks
  • Human in the loop
  • AI / model
  • System-of-record action

The process, in words

  • At rungs 1–2 a single feeder fault arrives as hundreds of alarm rows carrying equal visual weight. The operator performs the correlation in their head, at speed, from rows — and the quality of the first action depends on which experienced individual happens to be on shift.
  • At rungs 3–4 the same fault arrives against a rationalised alarm set. A correlation layer groups it into one event with ranked cause candidates, a suggestion service attaches provenance, confidence and an expiry, and the result renders inside the alarm summary and on the mimic. The operator authorises, edits or rejects — and every rejection returns to the system as a reason code.
  • At rung 5 a short, enumerated set of bounded actions executes inside a versioned envelope with a tested inhibit. Every execution is annunciated on the console with alarm-grade prominence, and anything outside the envelope stops and escalates to the desk rather than proceeding on a guess.
Step-by-step insights
Why one fault produces hundreds of rows
An electricity alarm flood is not a coincidence of unrelated events; it is one physical event observed by hundreds of instruments. A single earth fault operates protection, changes switch positions, trips downstream reclosers, drops voltage at connected substations, alarms low-voltage monitors, and raises communications alarms as devices lose supply. Every one of those is a true, correctly configured alarm. This is precisely why suppression is the wrong first instinct and correlation is the right one: nothing here is noise in the signal-processing sense — it is one story told by four hundred witnesses in arbitrary order.
The rationalised alarm set is the substrate, not the ambition
Correlation only produces interpretable output if each row means something agreed. ANSI/ISA-18.2 gives the lifecycle — philosophy, identification, rationalisation, design, implementation, operation, maintenance, monitoring and management of change — and EEMUA 191 gives the design guidance and performance targets that the HSE quotes. The reason this matters to an AI programme is not compliance: it is that 'these forty alarms are consequential' is a claim that can only be checked if somebody once wrote down what each of those forty alarms was for.
Provenance is the difference between a suggestion and an assertion
A suggestion that cannot show its evidence will be checked once, found unverifiable, and ignored thereafter. Provenance in a control room is specific: which telemetry points, which switch positions, which protection operations, which measurements, and how old each of them is. Data age is the item most often omitted and the one operators ask for first, because a confident statement built on a three-minute-old scan during a fast-moving event is a different object from the same statement built on live values.
Rendering inside the console is an assurance question too
Putting anything on a live operator console is a change to a safety-relevant human-machine interface, which is why the HSE treats control-room and HMI design as human-factors topics in their own right, and why NERC CIP change control applies to systems in the cyber boundary. The practical consequence is a longer approval path than teams expect and a shorter development path than they fear: most of the elapsed time is assurance, training and simulator work, not engineering.
The override edge is the learning loop
The dashed arrow back from the operator to the suggestion service is the most valuable line on the diagram. Overrides with reason codes are how a programme discovers that its loading assumption is wrong, that a proposal ignores a customer commitment, or that its ordering is unsafe with current permits. Without that edge, a control-room AI programme has no evidence base and its next release is guesswork — which is why rung 4 is unreachable from a rung-3 system that never captured dismissals.
Rung 5 is a list, and the inhibit is the thing to look at
Supervised autonomy earns its name from what the operator retains: alarm-grade annunciation of every automated action, and a single inhibit that takes the scheme out of service without asking anyone's permission. If either is missing, the control room has traded away situational awareness for speed. The rate of escalations out of the envelope is the leading indicator to watch — a rise means the network has moved, and the envelope needs revalidating before an incident forces the point.

The five rungs in detail

For each rung: what it looks like on a real desk, the signals a reviewer can check in an afternoon, the anti-pattern that traps control rooms there, and what leaving costs.

Each rung below is written for a practitioner rather than a buyer. The hallmarks are observable conditions on a desk, the diagnostic signals are checks you can run against your own historian and change records this week, and the anti-pattern is the specific mistake most often made trying to leave that rung. The ladder is deliberately front-loaded: rungs 1 and 2 contain no AI at all and carry a large share of the total value.

Select a rung

Every rung's full detail is in the page source — the selector only changes which panel is visible, so nothing here depends on JavaScript to exist.

Stage 1

Alarm noise

24% of operators sit here

The console is saturated: alarms without defined responses, standing and chattering points, and floods that the operator survives on experience alone.

Rung 1 is not incompetence; it is accumulation. Every protection scheme commissioned, every new RTU point, every 'add an alarm for that' action item from an incident review adds rows to the same list, and nothing ever removes any. Over fifteen years the alarm system stops being a set of things that require a response and becomes an undifferentiated event stream that the operator is expected to interpret in real time.

The tell is the language the desk uses. At rung 1 operators talk about 'clearing the list' rather than about responding to alarms, and they will tell you — accurately, and without embarrassment — which alarms they ignore. That knowledge is real expertise, and it is also an uncontrolled risk register: it lives in individuals, it is not written down, and it walks out of the building with every retirement.

AI cannot be added usefully here, and adding it makes things worse in a specific way. A model trained on this alarm stream learns the noise as if it were signal, and any assist rendered on top of it either repeats the flood in a new format or hides parts of it by rules nobody agreed. The prerequisite is not a data platform; it is the unglamorous rationalisation work that the process industries have had a standard for since the 1990s.

In practice

The eleven minutes nobody could have handled

The Health and Safety Executive's account of the 1994 Texaco Milford Haven explosion is the reference case the whole alarm-management discipline is built on: in the last eleven minutes before the explosion, two operators had to recognise, acknowledge and act on 275 alarms. The plant was chemical rather than electrical, but the arithmetic is identical on a distribution desk during a storm — the human channel has a fixed bandwidth, and no amount of professionalism raises it.

What it looks like

  • Nobody can state the desk's steady-state alarm rate per operator without a special study
  • Standing alarms are permanently present and mentally filtered out
  • A single feeder fault produces hundreds of alarm rows in under a minute
  • Which alarms matter is knowledge held by long-serving operators, not by the system

Diagnostic signals you can check this week

  • Ask the desk what its alarm rate is per operator per ten minutes in normal operation. A shrug is the answer
  • Count standing alarms at the start of a shift; anything above single figures means the list is decorative
  • Look at the top ten most frequent alarm points over a month — chattering points usually dominate
  • Ask whether every alarm has a defined operator response written down. At rung 1 the honest answer is no

Anti-pattern · Buying an AI alarm-reduction product first

The obvious fix looks like a product: install something that suppresses or clusters alarms with machine learning and the noise problem disappears. It does not. Suppression rules that nobody rationalised become invisible policy — an alarm that a human decided mattered is now hidden by a model nobody can interrogate at three in the morning, and the first time a genuine event is inside a suppressed cluster, the tool is switched off permanently and the credibility of every later system goes with it. Rationalise first; the AI has to sit on top of an agreed set of alarms, not replace the agreement.

What holds you here

The alarm set has never been rationalised, so there is no agreed signal for an assist to reason over — only an event stream.

Highest-leverage next move

Measure one desk against the EEMUA 191 bands and rationalise the worst forty alarm points. No AI in this phase at all.

Cost of leaving

Effort
3–6 months
Team
One control engineer, one protection or SCADA engineer, and operator time from every shift
Risk
Low — the work removes and re-tunes alarms under change control; nothing new is added to the console
To next stage
3–6 months

If this is you, the next step is

Two to three weeks: rates by shift, standing and chattering inventory, flood characterisation from the historian.

Scope an alarm baseline for one desk

Stage 2

Rationalised

34% of operators sit here

Every alarm has a defined operator response, rates are measured against a published benchmark, and floods are characterised rather than endured.

Rung 2 is where most of the safety value on this whole ladder is actually released, and it involves no artificial intelligence whatsoever. Rationalisation is a documented method — ANSI/ISA-18.2 gives it a lifecycle, EEMUA 191 gives it design guidance and performance targets — and the effect on a saturated desk is immediate and measurable in a way that later, cleverer work rarely is.

It is also slow, and honest planning matters. The HSE's own guidance notes that a quick first-pass review may cover perhaps fifty alarms per shift while a thorough review and redesign may take more than one shift per alarm. On a desk with four thousand configured alarms that arithmetic is the entire programme plan, which is why serious utilities rationalise by risk-ranked tranches rather than attempting the whole estate.

What rung 2 produces for the AI work that follows is not data but agreement. After rationalisation, an alarm means something specific that operations, protection and engineering have all signed. That shared definition is what makes a correlation model's output interpretable — 'these forty rows are one event' is only a useful sentence if everyone agrees what each row was for.

In practice

Forty points, one storm, three fewer screens of scrolling

A distribution control desk pulled twelve months of alarm history from the historian and found that forty configured points produced more than half of all alarm rows, almost all of them chattering analogue crossings and out-of-service plant. Re-tuning deadbands, filtering the chatter and suppressing out-of-service plant under an expiring shelve reduced the steady-state rate below the EEMUA manageable band. No model was involved; the operators noticed within one shift.

What it looks like

  • An alarm rationalisation record exists: purpose, cause, consequence, response and priority per alarm
  • Steady-state rate per operator is measured continuously and reported like any other KPI
  • Shelving and suppression are explicit, time-limited and logged, not informal
  • Flood events are counted, timed and reviewed — the desk knows its worst ten minutes of the year

Diagnostic signals you can check this week

  • Ask to see the rationalisation record for a named alarm. Rung 2 produces it in a minute
  • Check whether shelved alarms have expiry times and whether anyone reviews the shelved list at handover
  • Look for a monthly alarm-performance report that reaches an accountable manager, not just the SCADA team
  • Ask how a new alarm gets added — a change-control route with an impact-on-load assessment is the rung-2 signature

Anti-pattern · Declaring rationalisation done and letting it decay

Alarm systems regress silently. Every commissioning, every temporary configuration left in place, every incident action item that adds 'an alarm so this cannot happen again' pushes the rate back up, and eighteen months later the desk is at rung 1 with a rationalisation record that describes a system nobody has operated for a year. The management-of-change requirement in the ISA-18.2 lifecycle exists precisely for this, and it is the part most often skipped once the initial project ends.

What holds you here

The alarm set is clean but nothing yet interprets it — the operator still assembles the picture from rows during the exact minutes when assembling is hardest.

Highest-leverage next move

Add correlation inside the existing alarm summary: group a flood into one event and offer ranked cause candidates with their evidence, changing nothing about who decides.

Cost of leaving

Effort
6–12 months for a full desk, less per tranche
Team
Control engineer, protection engineer, SCADA configuration, plus scheduled operator workshops
Risk
Medium — removing an alarm is a safety-relevant change and needs the same rigour as adding one
To next stage
6–12 months

If this is you, the next step is

We sequence tranches by contribution to load and by consequence, so the first tranche is felt on the desk.

Plan a risk-ranked rationalisation

Stage 3

Assisted

27% of operators sit here

AI adds interpretation inside the operator's own screens — flood grouping, cause candidates, 'what changed' summaries — while every decision stays exactly where it was.

Rung 3 is the first rung where AI earns its place, and its contribution is narrow and precise: it compresses the perception and comprehension work that Endsley's model of situational awareness describes as the first two levels, so that the operator can spend their attention on the third — projecting what happens next. Grouping four hundred alarm rows into 'one earth fault on this feeder, these are the consequential alarms, these three are not explained by it' is worth more than any prediction, because it hands back minutes at the exact moment the desk has none.

The engineering discipline that distinguishes a rung-3 system from a demo is provenance. Every statement the assist makes must be traceable to the telemetry that produced it, timestamped, and legible in one glance — which switch position, which protection operation, which measurement, and how old each of them is. An operator with personal accountability for a switching decision will not, and should not, act on an unattributed assertion, and a system that cannot show its evidence gets ignored inside a fortnight.

The other rung-3 requirement is silence. The assist must be able to say nothing. When telemetry quality degrades, when the state estimator has not solved, when the event does not match anything the model has seen, the correct output is an explicit 'insufficient evidence' rather than a low-confidence guess presented in the same visual language as a confident one. Abstention is a feature that operators notice and reward with trust.

In practice

Four hundred rows, one sentence

A regional desk takes an 11 kV fault during a storm. Before assist, the operator sees several hundred alarm rows across three screens and reconstructs the picture from experience. After assist, the top of the alarm summary reads: one event, circuit breaker operated at a named substation, protection indication consistent with an earth fault on a named section, twelve downstream low-voltage alarms explained by it, and two alarms not explained — highlighted, because unexplained alarms are the ones that matter. The operator still makes every decision. They make the first one four minutes earlier.

What it looks like

  • An alarm flood arrives as one grouped event with a ranked list of candidate causes and their evidence
  • The assist renders inside the alarm summary and on the single-line diagram, not in a separate application
  • Every suggestion carries provenance, a confidence statement and an expiry
  • Suggestion views, accepts and dismissals are logged at desk level with reason codes

Diagnostic signals you can check this week

  • Watch an operator during a flood: if they leave the alarm summary to consult the assist, integration has failed
  • Ask the assist to explain one suggestion. If provenance takes more than a glance to reach, it will not be checked under pressure
  • Check whether the assist has ever declined to answer, and whether operators can recall it doing so
  • Look for the override log. If dismissals are not captured with reasons, rung 4 has no evidence base to build on

Anti-pattern · Improving the model to fix adoption

When operators do not use the assist, the reflex is to make it more accurate. Adoption in a control room is overwhelmingly a function of location, latency and legibility rather than accuracy: a moderately good cause ranking on the alarm summary changes more decisions than an excellent one behind a second login. Before touching the model, measure how many keystrokes and how many seconds separate the operator from the suggestion and from its evidence — that number, not the F1 score, is what is capping use.

What holds you here

The assist explains the present but proposes nothing, so the operator still constructs every action sequence by hand under time pressure.

Highest-leverage next move

Move from interpretation to proposal: draft the switching or restoration sequence, checked against tags, permits and limits, for the operator to accept, edit or reject.

Cost of leaving

Effort
6–12 months
Team
Integration engineer with SCADA/ADMS experience, ML engineer, control-room human-factors input, named operations owner
Risk
Medium — anything rendered on a live console is a change to a safety-relevant interface and needs its own assurance
To next stage
9–18 months

If this is you, the next step is

Where the suggestion renders, how it is dismissed, and what happens when it is unavailable.

Design the console integration

Stage 4

Guided

12% of operators sit here

The assist proposes a next best action — a switching sequence, a restoration order, a relief action — checked against constraints, and the operator authorises, edits or rejects it.

Rung 4 changes the artefact. Instead of describing the network, the assist proposes an action on it, and that crosses a line the industry has always taken seriously: a switching programme is a controlled document, prepared and checked by authorised people, and a restoration order commits real customers to real outage minutes. A proposal is legitimate at this rung only if it arrives inside the existing preparation workflow, carries its constraint checks visibly, and is trivially rejectable.

The evidence engine of rung 4 is the override log built at rung 3. Acceptance rate by proposal type is the honest measure of whether the guidance is useful, and the reason codes attached to rejections are worth more than the acceptances: 'unsafe with current permits', 'ignores a customer commitment', 'right answer, wrong order' each point at a different defect, and the distribution across a quarter is the most reliable roadmap a team will get.

Training is a hard requirement rather than a nice-to-have here. NERC's PER-005-2 already obliges reliability coordinators, balancing authorities and transmission operators to train system operators through a systematic approach, and requires emergency-operations training using simulation technology where interconnection reliability operating limits are in play. A new decision aid that appears on the live console without ever having appeared in the simulator is, in that framework, an untrained tool in a trained environment.

In practice

The restoration order that arrives already checked

After a fault is isolated, the ADMS proposes three restoration sequences, ranked by customers restored per step and annotated with the constraint each one nearly violates: option A restores 1,400 customers in two steps but loads a transformer to 96% of rating on the forecast evening peak; option B restores 900 with headroom; option C waits for a field confirmation. The control engineer picks B, edits one step, and the edit is logged. Three months later, that edit pattern — repeated by four operators — is what tells the team the loading assumption was wrong.

What it looks like

  • Draft switching and restoration sequences appear in the tool that already builds them, pre-checked against tags and permits
  • Each proposal states what it assumes, what it would achieve, and what it does not know
  • Acceptance and override rates are monitored per proposal type, with reason codes reviewed monthly
  • Operators are trained on the assist in the simulator, including on deliberately degraded behaviour

Diagnostic signals you can check this week

  • Check whether the proposal is pre-checked against live tags, permits and earths, or whether the operator has to do that themselves
  • Read a month of override reason codes; an empty or single-value field means nothing is being learned
  • Ask whether the assist has ever been exercised in the simulator with degraded or wrong inputs
  • Ask an operator what happens if they reject every suggestion for a shift. If the answer involves anyone being asked why, the design is already coercive

Anti-pattern · Measuring the operator instead of the desk

Once acceptance data exists, someone will propose per-operator dashboards: who accepts, who overrides, who is slower. It is the fastest way to destroy a control-room programme. Operators hold personal, certified accountability for their decisions, and an instrument that appears to grade their judgement converts a decision aid into a surveillance tool — after which the honest overrides stop and the reason codes turn into whatever is safest to type. Aggregate to desk, shift type and proposal type; write the prohibition into the measurement plan and agree it with the operators' representatives before go-live.

What holds you here

Every action still waits for an operator keystroke, so throughput in a mass-event scenario is bounded by how many decisions a desk can physically authorise.

Highest-leverage next move

Define the narrow, enumerated envelope in which a bounded action may execute unattended, with a tested inhibit, and prove it in the simulator before it is armed on the network.

Cost of leaving

Effort
12–24 months
Team
Platform and integration team, control engineering, simulator and training lead, safety-rules authority, human factors
Risk
Higher — proposals touch controlled documents, so assurance, change control and training evidence become the binding constraints
To next stage
18+ months

If this is you, the next step is

From telemetry to constraint check to console to override log — where the evidence gaps are.

Review a proposal path end to end

Stage 5

Supervised autonomy

3% of operators sit here

A small, enumerated set of bounded actions executes without per-action approval inside a versioned envelope, with the operator supervising, inhibiting and handling everything outside it.

Rung 5 is much narrower than the phrase suggests. It is not an unattended control room; it is a handful of decisions — fault isolation and restoration on a defined feeder class, dispatch of a pre-optimised set of balancing instructions, a Volt/VAR setpoint inside stated limits — that execute without waiting for a keystroke, while everything else stays exactly where it was. Utilities that describe rung 5 well always describe it as a list, never as a capability.

The operator's role changes shape rather than shrinking. Supervision means seeing what the system did as clearly as what it proposes, holding a working inhibit, and retaining the authority to take manual control without asking anyone. If the automated action is not annunciated on the same console with the same prominence as an alarm, the control room has lost situational awareness in exchange for speed, which is the single worst trade available in system operation.

Sustaining rung 5 is governance, not engineering. Envelopes drift out of validity as the network changes — new generation, new customers, a reconfigured feeder — and the evidence that satisfied last year's assurance review is not automatically adequate this year. The discipline that keeps it honest is the same one NERC applies to real-time monitoring in IRO-018: you must be told when the machinery you depend on has stopped working, because silent failure of a trusted system is worse than a visible one.

In practice

Bounded, annunciated, inhibitable

A distribution operator arms automated fault isolation and restoration on a defined class of feeders. Each execution posts to the alarm summary as a first-class event — what operated, what it restored, what remains isolated — and any case that falls outside the envelope stops and escalates to the desk with the reason. The control engineer keeps a single inhibit that takes the whole scheme out of service and does not require anyone's permission to use. Escalation frequency is reviewed monthly: a rise means the network has moved outside the envelope, not that operators are being difficult.

What it looks like

  • The autonomous action set is written down, short, and reviewed like a safety document
  • An operating envelope states the conditions under which each action may execute, with a tested inhibit
  • Every automated action is annunciated to the desk and reconstructable from the decision log
  • The rate of actions falling outside the envelope is monitored as a leading indicator

Diagnostic signals you can check this week

  • Ask for the list of actions permitted to execute unattended. It should be short and printed
  • Ask when the inhibit was last exercised deliberately, and whether the exercise was recorded
  • Check whether automated actions appear on the operator's console with alarm-grade prominence
  • Ask who reviews the envelope, how often, and what triggers a review outside the cycle

Anti-pattern · Extending the envelope by analogy

The envelope is proven on one feeder class, works well, and is then extended to a class that was never in the evidence — different protection philosophy, different customer mix, different telemetry quality. The first bad automated action results in the entire scheme being disarmed, and the control room's tolerance for anything automated drops for years. New action types re-earn autonomy from their own evidence, in the simulator first, with their own envelope and their own inhibit.

What holds you here

Sustaining autonomy is an assurance discipline: envelopes age as the network changes, and the evidence has to be rebuilt rather than inherited.

Highest-leverage next move

Treat the envelope as a versioned, reviewed safety artefact with named owners, scheduled revalidation and a drilled inhibit.

Cost of leaving

Effort
Continuous
Team
Platform team, control engineering, safety and assurance, plus a standing review forum including operators
Risk
Concentrated — low frequency, high consequence, and squarely inside the regulator's field of view

If this is you, the next step is

We run the envelope, the inhibit and the annunciation against a real scenario with your operators.

Stress-test an autonomy envelope

Where control rooms actually sit on the ladder

The distribution across the five rungs, and why the largest transition loss is between a rationalised alarm system and a genuinely assisted one.

Most control rooms are on rungs 1 and 2 — that is, in alarm-management territory rather than AI territory. The distribution below is illustrative rather than surveyed: it is our synthesis of what the published alarm-management literature, the human-factors regulators and the energy research houses describe, and it is drawn to make one point that every number in it supports — the mass of the industry sits below the rung where a model does anything.

Illustrative distribution of electricity control rooms across the five rungs

Illustrative, not surveyed. Rungs 1 and 2 dominate: the binding constraint in most control rooms is alarm quality and console integration, not model capability. The steepest drop is 2 → 3, where interpretation has to move from the operator's head into the console without adding anything to the screen.

Share of control rooms

  • 24% — 1 · Alarm noise
  • 34% — 2 · Rationalised (the plateau)
  • 27% — 3 · Assisted
  • 12% — 4 · Guided
  • 3% — 5 · Supervised autonomy

Source: Illustrative distribution, synthesised from IEA, HSE and ISA/EEMUA alarm-management material

The external picture is consistent. The IEA's Energy and AI report (opens in a new tab) frames AI's grid contribution largely in terms of faster fault identification and better use of existing assets rather than new autonomy; the HSE's human-factors guidance (opens in a new tab) has treated alarm handling, control-room design and shift handover as first-order safety topics for two decades; and the alarm-management standards themselves — ANSI/ISA-18.2 (opens in a new tab) and EEMUA 191 (opens in a new tab) — exist because the industry established, expensively, that operator attention is the scarce resource. Research bodies including EPRI (opens in a new tab) continue to publish on control-centre modernisation for the same reason.

Alarm rationalisation and flood management: the work before the AI

The published benchmarks, why an electricity flood is one event told by hundreds of witnesses, and the lifecycle that keeps a rationalised system rationalised.

Alarm rationalisation is the process of deciding, alarm by alarm, whether a signal deserves to interrupt a human being — and if it does, what the operator is supposed to do about it, how urgent it is, and how long they have. It is the prerequisite for every rung above it on this ladder, because an assist that reasons over an unrationalised alarm set inherits its incoherence and presents it with more authority.

Alarm rate per operatorPublished bandWhat the operator can actually doWhat assist may legitimately doRung
Under 1 per 10 minutesVery likely to be acceptableRead every alarm, respond to each on its merits, keep a mental model of the networkAdd projection: time-to-limit, forecast constraint, quiet context for the next shift3–4
1–2 per 10 minutesManageableKeep up in normal operation; slip during concurrent eventsGroup consequential alarms; annotate the mimic; draft the routine switching sequence3–4
2–10 per 10 minutesOver-demandingTriage rather than respond; some alarms are acknowledged without being readOnly interpretation — grouping and cause ranking. No proposals until the rate comes down3
More than 10 per 10 minutesUnacceptableSilence and scroll; the alarm system has stopped functioning as an alarm systemNothing on the console. The work is rationalisation, not inference1–2
Flood after a major eventNo more than 10 displayed in the first 10 minutes is the design targetReconstruct the event from rows while making the first decisionsGroup the flood into one event with ranked causes and explicitly unexplained alarms3
Alarm-rate bands as published by EEMUA 191 and quoted in the HSE's information sheet on better alarm handling, read against what an assist can legitimately contribute at each band. The bands are process-industry benchmarks and transfer directly to electricity control desks, where the flood mechanism is the same.

Those bands come from EEMUA Publication 191 (opens in a new tab), the alarm-systems guide the HSE quotes in its own guidance (opens in a new tab) on better alarm handling: a long-term average alarm rate in normal operation of no more than one every ten minutes, and no more than ten alarms displayed in the first ten minutes following a major upset. They were written for process plant, and they transfer to electricity control desks without amendment because the constraint they describe is the operator, not the plant.

1 / 10 min

Long-term average alarm rate per operator that EEMUA 191 treats as the design target in normal operation

EEMUA 191, quoted by HSE

10 / 10 min

Maximum alarms displayed in the first ten minutes after a major upset, as a design target

EEMUA 191, quoted by HSE

> 1 shift

Time a thorough review and redesign of a single alarm can take; a quick first pass covers about 50 per shift

HSE, Better alarm handling

In the last 11 minutes before the explosion the two operators had to recognise, acknowledge and act on 275 alarms.

The electricity-specific point is that a flood is not noise. One earth fault operates protection, changes switch positions, trips downstream devices, drops voltage at connected substations, alarms low-voltage monitors, and raises communications alarms as equipment loses supply. Every row is a true alarm about a real state change. That is why the correct first move is correlation — collapsing one event into one line with its consequences attached — and why blanket suppression is the move that eventually hides something that mattered.

The rationalisation lifecycle, applied to one distribution desk

  1. Write the philosophy before touching a point

    A short, signed document that states what an alarm is on this system, the priority definitions and roughly how they should be distributed, the performance targets taken from EEMUA 191 (opens in a new tab), and who may add one. The ISA-18.2 lifecycle (opens in a new tab) starts here for a reason: without it, rationalisation becomes an argument per alarm.

  2. Measure before you change anything

    Pull twelve months from the historian: rate per operator per ten minutes by shift, standing alarms at shift start, the top fifty most frequent points, and every flood over the design target with its duration. This baseline is the only thing that will later prove the work was worth doing.

  3. Rationalise by risk-ranked tranche

    Take the points that contribute most to load first. For each: purpose, cause, consequence of no action, the defined operator response, the time available, and the priority that follows from the consequence. The HSE's guidance is blunt about the effort involved — plan in tranches, not in one campaign.

  4. Fix the mechanical causes

    Deadbands on repeating analogue crossings, filters on chattering points, suppression of alarms from out-of-service plant under an expiring shelve, and removal of alarms with no defined response. This is where the rate drops fastest and where operators notice within a shift.

  5. Put management of change around it

    A new alarm requires the same route as any other change to a safety-relevant interface: a stated response, a priority derived from consequence, and an assessment of its effect on the desk's alarm load. Without this step the system regresses to rung 1 within two years, which is the single most common outcome of a successful rationalisation project.

  6. Report performance where it will be read

    Monthly alarm performance — rate, standing alarms, floods, top contributors — to an accountable operations manager rather than to the SCADA team alone. Alarm systems decay when nobody with authority is looking at the number.

Only after this is there something for a model to work with. The order is not a preference: a correlation layer trained on chattering points learns that they are informative, and a cause ranker built on alarms with no defined response ranks candidates nobody can act on. Rationalisation is the cheapest capability on the ladder and the one that makes every later pound of AI spend defensible.

The control-room moment map: where assist belongs and where it does not

Eight moments an operator lives through, the assist that is legitimate in each, the console surface it must render on, and the outcome it should move.

Assist belongs to moments, not to systems. A control room is not a continuous state; it is a sequence of quite different situations — quiet monitoring, planned switching, a fault, a storm, a handover — each with its own attention profile, its own time budget and its own tolerance for a machine offering an opinion. The map below is how we scope assist work with utilities: pick a row, deliver that row properly on one desk, and only then pick another.

Control-room momentWhat the operator is doingLegitimate assistWhere it must renderEarliest rungOutcome measure
Steady-state monitoringScanning, maintaining a mental model, absorbing routine alarmsSuppression of out-of-service plant, chatter filtering, shelving with expiry, quiet 'what changed' summariesAlarm summary and the shelved-alarm list2Rate per operator per 10 min; standing alarms at shift start
Alarm flood after a faultTriaging hundreds of rows while making the first decisionsGroup the flood into one event; rank cause candidates with evidence; highlight alarms not explained by the leading candidateTop of the alarm summary, as a grouping of existing rows3Alarms presented per flood; time to first correct action
Fault location and restorationDeciding the isolation and restoration sequenceRanked restoration options with customers restored per step and the constraint each nearly breachesSingle-line diagram annotation and the OMS incident4CML/CI or SAIDI/SAIFI on covered feeders; restoration time
Planned switchingPreparing and checking a switching programmeDraft step sequence pre-checked against live tags, permits and earths; conflicting-permit detectionThe switching schedule editor already in use4Preparation time; defects found at check stage
Constraint and limit managementWatching a boundary or a transformer approach its ratingTime-to-limit projection with the assumptions stated, plus candidate relief actionsTrend and limit panel beside the value being watched3Time spent in alert state; number of exceedances
Storm and major-event operationRunning many concurrent incidents with degraded telemetryEvent clustering, damage prediction, prioritisation by customers and vulnerability, ETR supportEvent list and geographic view3Events per operator; estimated restoration time accuracy
Shift handoverBriefing the incoming shift on abnormalities and intentAuto-drafted handover pack: open abnormalities, shelved alarms with expiry, active suggestions and overridesThe handover record itself3Handover completeness audit; repeat questions after handover
Post-event reviewReconstructing what happened and whyEvent timeline with decisions, data quality at the time, and the assist's own provenanceEvent journal and historian replay3Time to reconstruct an event; findings per review
The control-room moment map. 'Earliest rung' is the ladder position at which each assist becomes legitimate rather than merely possible — proposals before a rationalised alarm set, or on a second monitor, fail regardless of model quality.

Read down the fourth column and the page's central claim becomes concrete: every legitimate assist renders on a surface the operator already looks at. Nothing in the map is delivered as a new application, and the two rows with the highest stakes — restoration and planned switching — are the two that must arrive inside controlled documents the utility already governs. Where those proposals are computed by an optimiser rather than a learned model is an implementation detail covered on the grid and layout optimisation page; what matters here is that the output lands in the switching schedule with its constraint checks visible.

  • Compress, don't add

    The first family of assist takes what is already on the screen and makes it smaller: grouping, deduplication, explanation. It is the only family that is unambiguously welcome during a flood, because it reduces the number of things demanding attention rather than increasing it.

  • Project, don't predict theatrically

    The second family answers 'when', not 'what if': time to a rating, time to a limit, time until a constraint binds. Projection is what Endsley's third level of situational awareness describes, and it is the level operators lose first under load. A projection with its assumptions rendered alongside it is trusted; a bare number is not.

  • Propose, with the checks attached

    The third family drafts an action. It is legitimate only when the proposal arrives pre-checked against the constraints the operator would check anyway — tags, permits, earths, ratings, customer commitments — and only when rejecting it costs one keystroke. A proposal that is expensive to reject is a proposal that will be accepted for the wrong reason.

  • Record, because someone will ask

    The fourth family produces no output for the operator at all: it writes the decision log — every suggestion made, seen, accepted, edited or rejected, with reasons and data ages. It costs almost nothing to build alongside the other three and it is the only reason the programme will survive its first post-incident review.

What operator assist looks like in public

Three publicly reported programmes, read against the ladder. None is an Atomic Loops engagement — each links to the operator's own published material.

The clearest public evidence for the argument on this page is in what system operators chose to build. In each programme below, the differentiator is not the sophistication of the model but the fact that the output arrives where the operator already works — a dispatch list inside the balancing tools, an automated restoration annunciated to the desk, a real-time renewable picture inside the national control centre — and that the surrounding discipline was built at the same time.

Three programmes read against the ladder

Outcomes as reported by the operators themselves; verify figures against the linked source before reusing them, as we have not independently audited them. The card images are generated library illustrations of control-room scenes, not photographs of these operators or their facilities, and no endorsement is implied.

Illustrative scene: system operators reviewing a national network model on a control-centre displayNational Energy System Operator (NESO)GB electricity system operator · national balancing control room34
Challenge
Balancing a system with rapidly growing numbers of small balancing mechanism units and battery storage sites meant control-room engineers had to instruct each asset individually, which limited how much of that flexible capacity could realistically be used second by second.
Approach
The Open Balancing Platform introduced Bulk Dispatch: the platform presents control-room engineers with a pre-selected, optimised list of units that meets a network requirement, and lets them instruct hundreds of smaller units and battery sites in a single action — inside the balancing workflow rather than as a separate analytical tool.
Reported outcome
NESO reports that the tool greatly reduces the time taken to instruct balancing mechanism units and the number of manual instructions required from the control room, with further stages delivered across 2024 and 2025 and the platform set to replace the existing balancing systems by 2027.
What it shows about the curveThis is rung 4 in its clearest form: the machine narrows the option set and prepares the action; the control-room engineer still chooses and commits it. The value came from putting the optimised list inside the dispatch path, not from the optimisation being novel.

NESO — first stages of the Open Balancing Platform go live (opens in a new tab)

Illustrative scene: a field engineer reviewing network diagnostics on a tablet beside transmission infrastructureDuke Energy FloridaUS investor-owned utility · distribution network, hurricane-exposed45
Challenge
Storm restoration on a large distribution network is bounded by how quickly faults can be located, isolated and switched around — work that, done manually, competes with everything else a control room is doing during a major event.
Approach
Self-healing technology detects an outage and reroutes power automatically on equipped parts of the network, isolating the faulted section without waiting for a per-action decision from the control room — a bounded, enumerated action set operating on a defined class of circuits.
Reported outcome
Duke Energy reported that during Hurricanes Helene and Milton the technology prevented more than 300,000 customer outages and saved customers more than 300 million minutes of total outage time, describing it as isolating the cause of an outage and reducing the number of customers affected by up to 75%, often restoring power in less than a minute.
What it shows about the curveRung 5 done properly is narrow and specific: one class of circuits, one action type, executed within an envelope. It is also the rung where annunciation and the inhibit matter most — an automated restoration the desk cannot see is a loss of situational awareness bought with speed.

Duke Energy — self-healing technology during Helene and Milton (opens in a new tab)

Illustrative scene: two grid operators working with a real-time network and generation displayRed Eléctrica (Redeia)Spanish transmission system operator · national control centre23
Challenge
Integrating very high volumes of variable renewable generation requires the national control centre to know, continuously and in real time, what every significant renewable facility is doing — a situational-awareness problem long before it is an optimisation problem.
Approach
Cecre, the control centre for renewable energies, sits inside Cecoel, the electricity control centre, and receives real-time information every twelve seconds from generating facilities via the generators' own control centres, covering connection status, production and voltage at the connection point — then supports real-time analysis of the system state and the operating measures required.
Reported outcome
Red Eléctrica reports that Cecre allows the maximum renewable production to be integrated while maintaining supply quality and security, and states that in 2024 it integrated more than 98% of renewable generation at peninsular level.
What it shows about the curveSituational awareness is architecture, not decoration. The rung-3 lesson is that the assist layer is only as good as the telemetry cadence and the model beneath it — and that placing the renewable picture inside the existing control centre, rather than beside it, is what makes it operational.

Red Eléctrica — Cecre, the control centre for renewable energies (opens in a new tab)

The HMI integration problem: the assist has to live on the operator's screen

Why a second monitor is where control-room AI goes to die, which console surfaces can legitimately carry a suggestion, and what handover inherits.

The HMI integration problem is simple to state and expensive to ignore: during the minutes when assist is worth most, the operator's eyes are on the alarm summary and the single-line diagram, and anything that is not on those surfaces is not read. Every control-room AI programme that fails for 'adoption' reasons failed here first — not because operators are resistant, but because attention during an event is a fixed quantity allocated to the screens they are accountable for.

  • Attention is allocated before the event, not during it

    Operators build a scanning pattern over years and hold it under stress. A new panel competes with that pattern precisely when the pattern is most entrenched. The design consequence is that assist output must join an existing element — a row, an annotation, a field — rather than occupying new screen real estate.

  • Latency budgets are set by the decision, not by the model

    If a cause ranking arrives ninety seconds after the flood, the operator has already formed a hypothesis and the suggestion is now competing with their own judgement rather than informing it. Set the latency budget from the moment — seconds for a flood, minutes for a restoration plan, hours for a planned switching programme — and treat a miss as a functional defect.

  • Keyboard-first or it will not be used at speed

    Control-room work is keyboard work. A suggestion that requires a mouse journey to accept, dismiss or expand adds seconds at the worst possible moment, and the HSE's guidance on human-computer interaction (opens in a new tab) in safety-critical settings is unambiguous that interface friction is a safety issue, not a preference.

  • Degraded mode has to be visible

    When the assist is unavailable, stale or working on suspect telemetry, the console must say so where the suggestion would have been. Silent absence is the dangerous failure: operators calibrate to a system that is usually there, and an empty space reads as 'nothing to report' rather than 'not working'.

SurfaceWhat it may carryHard constraintFailure if ignored
Alarm summaryEvent grouping, ranked cause candidates, explicitly unexplained alarmsMust never add rows; it may only fold existing ones, and grouping must be reversible in one keystrokeThe assist becomes another alarm source during the exact minutes the operator is saturated
Single-line diagram / mimicAnnotation of the faulted section, affected customers, proposed isolation pointsAnnotation must be visually distinct from telemetered state and must never look like a confirmed indicationAn inferred state is read as a measured one, and a switching decision is taken on a guess
Switching schedule editorDraft step sequences pre-checked against tags, permits, earths and ratingsThe draft is an input to the controlled document, never the document; the checking role stays humanAssurance and safety-rules governance are bypassed, and the programme is stopped by the authority that owns them
Outage / incident recordRestoration options ranked by customers restored, with the constraint each nearly breachesEvery option carries its assumptions and data ages; the operator's chosen option and any edit are loggedPost-event review cannot reconstruct why an option was taken, and the programme loses its evidence base
Handover recordDraft handover pack: abnormalities, shelved alarms with expiry, active suggestions and overridesDrafted for the outgoing operator to edit and sign; never auto-published as factThe incoming shift inherits a machine's summary of a human's intent, which is how context is lost
A separate application or second monitorAnalysis, tuning, model monitoring, post-event explorationNothing time-critical. This surface is for engineers and reviews, not for the live decisionThe whole programme reads as unused, and the conclusion drawn is 'operators rejected AI'
Console surfaces that can legitimately carry assist output, the constraint each imposes, and the failure that follows from ignoring it. The last row is the anti-pattern, included because it is still the most common delivery choice.

Shift handover deserves its own mention because it is where assist quietly pays for itself. The HSE treats shift handover as a distinct human-factors topic (opens in a new tab) precisely because incidents cluster around it: the incoming operator inherits a network state, a set of abnormalities and — critically — the outgoing operator's intent. A system that has been logging every suggestion, override and shelved alarm with an expiry can draft an accurate handover pack in seconds. It must be drafted for a human to edit and sign, never published as fact, because what handover actually transfers is intention, and no model has access to that.

Trust calibration: over-trust and under-trust both destroy the value

The two failure directions, the design variable that separates them, and the failure modes that send a working assist backwards.

Trust calibration is the work of making an operator's confidence in the assist match its actual reliability, case by case — and it fails in two directions that look nothing alike but cost the same. Over-trust turns a certified operator into someone who confirms without checking; under-trust turns an expensive programme into a panel nobody reads. The human-factors literature has described this since Parasuraman and Riley's work on the use, misuse, disuse and abuse of automation, and control rooms exhibit both halves within the same year.

Where trust actually lands

Plot how accurate the assist has been in the operator's own experience against how much it costs them to check it. Only one quadrant produces calibrated trust, and the lever that moves a control room into it is verification cost — a design variable, not a training one.

Calibrated trust

  • Operator checks because checking is nearly free
  • Confidence tracks reliability case by case
  • Target quadrant: keep verification cheap as the system grows

Drift into over-trust

  • Usually right, expensive to verify — so verification stops
  • Automation bias: the suggestion becomes the decision
  • Fix: bring provenance onto the same screen, not more training

Healthy scepticism, thin value

  • Operator checks and overrides correctly
  • Safe, but the assist is adding a step rather than removing one
  • Fix: model and data work — this is the one case where accuracy is the constraint

Abandoned

  • Unreliable and costly to check, so it is ignored
  • Usually misdiagnosed as cultural resistance
  • Fix: integration and provenance first, then accuracy; expect to rebuild credibility slowly
Accuracy the operator has personally seen — top: Right, and the misses were explained, bottom: Wrong often enough to remember
Cost of checking the suggestion — left: One glance, same screen, right: Another tool, re-derive it
  • Make checking cheaper than believing

    The single highest-leverage design decision is putting the evidence one keystroke from the suggestion: which points, which protection operations, which measurements, and how old each is. Over-trust is rarely a character failure; it is what rational people do when verification is expensive and the system is usually right.

  • Let the assist be wrong out loud

    Systems that decline to answer on insufficient evidence, and that visibly flag degraded telemetry, calibrate trust faster than systems that are marginally more accurate. Operators are experts in unreliable instruments — an instrument that admits its own limits is one they already know how to use.

  • Train on the failures, in the simulator

    NERC's PER-005-2 operations personnel training standard (opens in a new tab) already requires a systematic approach to training and, where interconnection reliability operating limits apply, emergency-operations training using simulation technology. Put the assist in those scenarios and include runs where it is wrong, stale or absent. An operator who has seen it fail in the simulator will not be surprised by it failing at 03:00.

  • Never reward acceptance

    The moment acceptance rate becomes a target, it stops being a measurement. Rates are read at desk and proposal-type level to find defects in the suggestion, never at operator level to find defects in the operator — and that prohibition belongs in writing in the measurement plan.

Likelihood: mediumImpact: high

Over-trust: the suggestion becomes the decision

The assist is right for months, verification is slightly awkward, and checking quietly stops. The failure surfaces on the one occasion the suggestion is confidently wrong — usually when telemetry was stale or the situation was outside anything the model has seen — and the operator's own network model has decayed from disuse.

PreventionProvenance and data age rendered beside every suggestion, plus deliberate degraded-assist scenarios in simulator training.

Likelihood: highImpact: medium

Under-trust: quiet abandonment

Nobody says anything. The suggestion panel is present, the programme reports 'deployed', and the accept rate drifts toward zero because the output arrives late, on the wrong surface, or without evidence. It is usually reported upward as cultural resistance, which sends the next investment to the wrong place.

PreventionTrack view and accept rates per suggestion type at desk level; treat a falling rate as an integration defect and open the override reason codes first.

Likelihood: mediumImpact: high

The assist adds noise during a flood

A system that generates its own notifications, banners or alerts becomes another alarm source at the worst possible moment. This is the fastest way to have assist switched off permanently, because it converts a decision aid into a contributor to the problem it was bought to solve.

PreventionArchitectural rule: the assist may fold, group and annotate existing rows, but may never create a new attention-demanding element during an event.

Likelihood: mediumImpact: high

Silent failure of the assist itself

The suggestion service stalls, the feed goes stale, or the model server is quietly restarted, and the console shows nothing where a suggestion would have been. Operators read absence as 'nothing to report'. NERC's IRO-018 requires an alarm process monitor for exactly this reason on the alarm processor; a trusted decision aid needs the same treatment.

PreventionA heartbeat monitor that notifies the desk — not just the platform team — plus a visible degraded-mode banner and a drilled fallback.

Likelihood: lowImpact: high

Accountability inversion

In the post-event review, the sentence 'the system recommended it' appears. It is a governance failure rather than an operator failure: it means the operating procedure, the training and the interface language collectively implied that the suggestion carried authority it does not have.

PreventionAdvisory wording on every proposal, unchanged accountability text in the operating procedure, and a decision log that records the operator's stated basis rather than only the system's.

Likelihood: mediumImpact: high

Measurement turns into surveillance

Acceptance and timing data get sliced by individual, a league table appears, and honest overrides stop within a fortnight. The data does not merely become useless; it becomes actively misleading, because the reason codes now record what is safest to type rather than what happened.

PreventionDesk, shift-type and proposal-type aggregation only, written into the measurement plan and agreed with the operators' representatives before go-live.

The assist architecture, layer by layer

What actually has to exist behind a suggestion — and which layer teams skip, every time.

A rung-3 assist needs six layers, and the order in which they are built decides whether the programme compounds or stalls on the console. The architecture below is deliberately vendor-neutral: every layer is defined by what it must guarantee to a certified operator, not by which product supplies it, and each is annotated with the rung that first requires it.

Layers required by rung

A programme trying to reach rung 3 without the console-integration and assurance layers is building a rung-2 alarm system with a model attached. The suggestion service is the layer most often built first and the one that matters least on its own.

  1. Telemetry and network model

    Stage 1+

    • SCADA / RTU / IEC 61850Measurements, statuses and protection indications, with scan rates
    • EMS / ADMS stateTopology, state estimation, load allocation — the model the operator trusts
    • HistorianThe event record every baseline and post-event review is built from
  2. Alarm and signal conditioning

    Stage 2+

    • Rationalisation recordPurpose, cause, consequence, response and priority, per alarm
    • Suppression and shelvingExplicit, time-limited, logged, visible in the shelved list
    • Data-quality flagsStale, suspect and manually entered values marked at source
  3. Event correlation and inference

    Stage 3+

    • Flood groupingOne physical event, its consequential alarms and its unexplained ones
    • Cause candidatesRanked, each with the evidence that supports and contradicts it
    • Limit projectionTime-to-rating and time-to-limit with assumptions stated
  4. Suggestion service

    Stage 4+

    • Candidate actionsPre-checked against tags, permits, earths, ratings and commitments
    • Confidence and expiryEvery suggestion states how sure it is and when it stops being valid
    • Explicit refusal'Insufficient evidence' as a first-class output, not a low score
  5. Console integration

    Stage 3+

    • In-place renderingAlarm summary, mimic annotation, schedule draft — never a new window
    • Keyboard interactionAccept, reject and expand evidence without leaving the keyboard
    • Degraded modeVisible banner where the suggestion would be, plus a defined fallback
  6. Decision log and assurance

    Stage 3+

    • Full decision trailEvery suggestion shown, seen, accepted, edited or rejected, with reasons
    • Heartbeat monitorThe desk is told when the assist stops, on the IRO-018 alarm-process pattern
    • Change controlModel, threshold and interface changes governed like any console change

Pipeline described

  1. Telemetry and network model (stage 1+) — SCADA / RTU / IEC 61850: Measurements, statuses and protection indications, with scan rates; EMS / ADMS state: Topology, state estimation, load allocation — the model the operator trusts; Historian: The event record every baseline and post-event review is built from
  2. Alarm and signal conditioning (stage 2+) — Rationalisation record: Purpose, cause, consequence, response and priority, per alarm; Suppression and shelving: Explicit, time-limited, logged, visible in the shelved list; Data-quality flags: Stale, suspect and manually entered values marked at source
  3. Event correlation and inference (stage 3+) — Flood grouping: One physical event, its consequential alarms and its unexplained ones; Cause candidates: Ranked, each with the evidence that supports and contradicts it; Limit projection: Time-to-rating and time-to-limit with assumptions stated
  4. Suggestion service (stage 4+) — Candidate actions: Pre-checked against tags, permits, earths, ratings and commitments; Confidence and expiry: Every suggestion states how sure it is and when it stops being valid; Explicit refusal: 'Insufficient evidence' as a first-class output, not a low score
  5. Console integration (stage 3+) — In-place rendering: Alarm summary, mimic annotation, schedule draft — never a new window; Keyboard interaction: Accept, reject and expand evidence without leaving the keyboard; Degraded mode: Visible banner where the suggestion would be, plus a defined fallback
  6. Decision log and assurance (stage 3+) — Full decision trail: Every suggestion shown, seen, accepted, edited or rejected, with reasons; Heartbeat monitor: The desk is told when the assist stops, on the IRO-018 alarm-process pattern; Change control: Model, threshold and interface changes governed like any console change
Step-by-step insights
Telemetry and model — the assist inherits the network model's errors
Everything above this layer is conditional on the topology and state the EMS or ADMS believes in. A connectivity error in the network model, a mis-mapped phase or an unmodelled temporary configuration will produce a confident, wrong cause ranking, and the operator will trace the error back to the assist rather than to the model. Before shipping any inference, sample a month of events and check the assist's inputs against what the desk actually saw — the discrepancy rate is the ceiling on everything else.
Conditioning — data quality has to travel with the value
A measurement that is stale, suspect, manually entered or frozen has a different evidential weight from a live telemetered one, and that distinction must survive all the way to the console. Most control-room AI disappointments trace back to this layer: an inference computed on values that were technically present and operationally meaningless. Flag quality at source, propagate it, and render it beside the suggestion.
Correlation — the unexplained alarms are the product
Grouping four hundred rows into one event is useful; identifying the two alarms the leading hypothesis does not explain is what changes decisions. Those are the ones that indicate a second event, a protection maloperation or a wrong assumption, and they are precisely what a saturated human misses. Design the output around residuals rather than around the headline cause and the assist becomes something operators seek out.
Suggestion service — refusal is a feature with a build cost
Teams routinely build confidence scores and rarely build abstention, because abstention requires an explicit model of what the system does not cover: unseen topologies, missing telemetry, event types outside the training distribution. It costs real engineering. It is also the single behaviour that most reliably earns operator trust, because it converts the system from an opinion generator into an instrument with a stated range.
Console integration — the layer that determines whether any of this is used
This is the layer teams defer to 'phase two' and the layer that decides the programme. It is also where the assurance work lives: rendering on a live console is a change to a safety-relevant interface, so it needs human-factors review, operator involvement in design, simulator exposure and change control. Budget it as a first-class workstream with its own approvals, not as front-end work at the end of a model project.
Decision log and assurance — build it first, use it forever
The decision log is cheap while the system is being built and impossible to reconstruct afterwards. It answers the post-event question ('what did the operator see, and when?'), the improvement question ('which proposals get rejected, and why?'), and the regulatory question ('can you show how this decision was reached?'). Pair it with a heartbeat monitor on the pattern NERC requires for alarm processors: you must be told when the thing you depend on has stopped.

The layer most often skipped is console integration, and the layer most often over-built is the suggestion service. The corrective is unglamorous: specify the console behaviour first — surface, latency budget, keystrokes, degraded mode — and let that specification constrain the model work, rather than building a model and then negotiating for screen space with the people who own the console.

Measuring operator outcomes without measuring the operator

Desk-level and event-level measures that answer the funding question, and the line that must not be crossed with certified professionals.

You measure the control room, not the operator — and the distinction is both an ethical requirement and the thing that keeps the data honest. Control-room staff hold personal, certified accountability for their decisions; an instrument that appears to grade individual judgement converts a decision aid into a performance-management tool, after which honest overrides stop, reason codes become defensive, and the programme loses the only evidence it had.

MeasureHow it is readUnit of analysisCadenceHonest from rung
Steady-state alarm rateAlarm rows presented ÷ operator-minutes, against the EEMUA bandsDesk × shift typeContinuous, reported monthly1
Standing and chattering alarmsAlarms active at shift start; top contributors by count over 30 daysDeskWeekly1
Flood frequency and durationEvents exceeding the 10-in-10-minutes design target, with time to return under itDesk × eventPer event2
Alarms presented per floodRows the operator had to read after grouping ÷ raw rows generatedEventPer event3
Time to first correct actionFirst operator action on the correct plant − event timestamp, from the historianEvent × deskPer event3
Suggestion acceptance rateAccepted ÷ shown, split by suggestion typeSuggestion type × deskWeekly3
Override reason distributionReason codes captured at rejection, reviewed with the deskSuggestion typeMonthly3
Restoration and reliabilityCML/CI or SAIDI/SAIFI on covered feeders against a comparison groupFeeder groupMonthly and annually4
Assist availabilityHeartbeat uptime and time-in-degraded-mode, annunciated to the deskSystemContinuous3
Handover completenessAudit of handover records against open abnormalities and shelved alarmsShift transitionMonthly sample3
Instrumentation sheet for operator-assist outcomes. Every measure is read from telemetry or logs the utility already keeps, and every unit of analysis is a desk, a shift type, an event or a suggestion type — never a named individual.

Attribution needs a comparison, and in a control room the natural holdout is a desk or a feeder group rather than a person. Run the assist on one desk and keep a comparable desk on the previous process for a quarter; or arm it on one feeder class and compare CML and CI against a matched class. Seasonality and weather will otherwise take the credit or the blame — a mild quarter can manufacture an improvement that no system produced, which is exactly the kind of claim that collapses under the first serious review.

Three commitments make the measurement plan acceptable to the people it measures, and they belong in writing before any logging starts: aggregation to desk, shift type, event or suggestion type with no individual-level reporting; a stated retention period after which raw interaction records are deleted; and agreement with the operators' representatives, who in most utilities are a recognised trade union with a legitimate interest in anything that looks like performance monitoring. Regulators sharpen the same point from the other side — the EU's AI Act framework (opens in a new tab) treats AI used in the management and operation of critical infrastructure as a high-risk category with obligations for human oversight, logging and transparency, and treats workplace monitoring as a category of its own.

A 90-day plan for one desk, and the gate before it goes live

The rung 2 → 3 transition made concrete on one distribution control desk — grouped floods and ranked cause candidates inside the existing alarm summary — and the eight conditions that must hold before any of it faces a working operator.

Moving one rung takes about 90 days when it is scoped to one desk and one moment, and several years when it is scoped to a control-room modernisation programme. To make that concrete, the plan below runs the transition on a specific and very common problem: a distribution control desk where a single feeder fault produces several hundred alarm rows, and where storm days push the rate far past the point at which the alarm system is doing its job. The quarter contains no autonomy, no new applications, and — for the first six weeks — no machine learning at all.

Rung 2 → rung 3 on one distribution desk, in one quarter

One desk, one moment, one named owner. If a phase needs longer than its window, narrow the scope — fewer feeders, one shift pattern — rather than extending the plan.

  1. Days 1–15

    Baseline the desk and agree the measurement rules

    Pull twelve months from the historian: alarm rate per operator per ten minutes by shift, standing alarms at shift start, the fifty most frequent points, and every flood over the design target with its duration. Name a control engineering owner and an operations owner. Write the measurement plan — desk-level only, stated retention, no individual reporting — and agree it with the operators' representatives before a single interaction is logged.

    A measured baseline and a signed measurement plan

  2. Days 16–40

    Rationalise the worst forty points

    Deadbands on repeating analogue crossings, filters on chattering points, expiring shelves for out-of-service plant, and removal of alarms with no defined operator response — each through normal change control with a stated response and priority. No AI in this phase. Expect the steady-state rate to fall far enough that operators comment on it unprompted.

    Steady-state rate inside the manageable band

  3. Days 41–65

    Run correlation in shadow mode

    Group floods into events with ranked cause candidates and explicitly unexplained alarms — logged, not displayed. Replay against the twelve-month history and against live events, and review a sample weekly with operators: what would it have said, and was that right? Build the decision log and the heartbeat monitor in this phase, not later.

    A measured hit rate and a reviewed failure set

  4. Days 66–80

    Render in the alarm summary for one desk

    The grouping appears at the top of the existing alarm summary, reversible in one keystroke, with evidence and data ages one keystroke away and a visible banner when the assist is degraded or absent. Run the simulator scenarios, including runs where the assist is wrong and where it is unavailable, before it appears on the live console.

    Assist live on one desk, inside the console

  5. Days 81–90

    Attribute against a comparison desk

    Report alarms presented per flood, time to first correct action, and the override reason distribution against a comparable desk left on the previous process. Not model accuracy. This is the number that funds the second desk, and the reason codes are the specification for the next release.

    A desk-level delta and a defect list from overrides

The order that keeps the certificate intact

  1. Rationalise before you infer

    Six weeks of alarm work beats six months of modelling on a saturated desk, and it is the only sequence in which the model's output is interpretable. It is also the phase that buys the control room's willingness to try the next one.

  2. Shadow before you show

    Log what the assist would have said for at least three weeks and review it with operators before anything reaches the console. The review meeting is where trust starts, because operators see the failures before they see the successes.

  3. Render inside, or do not render

    If the alarm summary cannot carry the output this quarter, spend the quarter making that possible rather than shipping a separate window. A second monitor is not a stepping stone to console integration; it is a substitute that removes the pressure to do it.

  4. Measure the desk, never the operator

    Write the prohibition down before you build the logging, and show it to the people being logged. It costs nothing, it is the right thing to do, and it is what keeps the override reason codes honest enough to be useful.

The plan ends at a gate rather than at a launch. The go-live gate is the short list of conditions that must hold before a suggestion appears in front of a working operator, and it exists because the cost of getting this wrong is not a failed project but a control room that will not trust the next attempt either. Each item below is checkable in an afternoon, and none of them is about model quality.

Before assist appears on a live console

If you cannot tick all eight, the assist is not ready for the desk regardless of how it performs offline. Tick as you go — this list works without JavaScript.

0 of 8 ticked

Nothing ticked — and that is a legitimate starting point

Zero ticks almost always means the programme is still upstream of the console: the alarm set has not been rationalised and nothing has been measured. Do not start with a model. Run the first two phases of the 90-day plan above on one desk — baseline, then the worst forty points — and most of this list becomes reachable.

Two of these items are effectively regulatory rather than optional. Training on a new decision aid sits inside the systematic-approach-to-training obligations that apply to system operators — NERC's PER-005-2 (opens in a new tab) being the explicit North American example — and monitoring for silent failure follows the pattern NERC sets in IRO-018 (opens in a new tab), which requires an alarm process monitor so that operators are notified when the real-time monitoring alarm processor has failed. European operators reach the same requirements through the ENTSO-E network codes (opens in a new tab) and their national implementations, and through the human-factors expectations regulators apply to control-room design (opens in a new tab).

Glossary

Hover a term for its definition — or expand the map full screen. The full definitions are written out below.

Alarm flood
A period in which alarms arrive faster than an operator can process them — conventionally more than ten in ten minutes. In electricity operations a flood is usually one physical event reported by hundreds of instruments rather than many unrelated events.
Alarm rationalisation
The documented process of deciding, per alarm, its purpose, cause, consequence of no action, defined operator response, time available and resulting priority. It is the substrate every higher rung of operator assist depends on.
Standing alarm
An alarm that is permanently or near-permanently active, typically because a condition is known and unresolved. Standing alarms train operators to filter the alarm list mentally, which is the habit that makes a genuine alarm easy to miss.
Chattering alarm
An alarm that repeatedly activates and clears in quick succession, usually an analogue value crossing a threshold without an adequate deadband. Chattering points routinely dominate the top of a desk's alarm-frequency list.
Shelving
Temporarily removing an alarm from the operator's active list under an explicit, time-limited and logged mechanism, with an expiry that forces a decision. Distinct from suppression, which is designed into the system rather than chosen by the operator.
Cause candidate
A ranked hypothesis about what produced an event, presented with the evidence that supports and contradicts it. The operationally valuable output is often not the top candidate but the alarms it fails to explain.
Situational awareness
The operator's perception of what is happening, comprehension of what it means and projection of what happens next — the three-level model described by Endsley. Assist can compress the first two; the third is where the human's expertise concentrates.
Automation bias
The tendency to accept a system's output without independent verification because it is usually right and checking is costly. It is a predictable consequence of design rather than a character flaw, and it is countered by making verification cheap.
Trust calibration
Bringing an operator's confidence in an assist into line with its actual reliability, case by case. Both miscalibrations are expensive: over-trust removes the human check, under-trust removes the assist's value.
Next best action
A proposed operational action — a switching step, a restoration sequence, a relief action — presented for a human to authorise, edit or reject, with its constraint checks and assumptions visible.
Operating envelope
The versioned statement of conditions under which a bounded action may execute without per-action approval, together with the inhibit that removes it from service. The artefact that defines supervised autonomy at rung 5.
Alarm process monitor
A heartbeat or watchdog that tells operators when the real-time alarm processor has failed — required of reliability coordinators by NERC's IRO-018. The same pattern is what stops an operator assist from failing silently.

Frequently asked questions

The questions control-room, engineering and operations leads ask most often when scoping operator assist.

What is AI operator assist in an electricity control room?

It is decision support that runs on the operator's own console during live operation. In practice it groups an alarm flood into a single event, ranks probable causes with their evidence, projects when a limit will be reached, and at higher maturity drafts switching or restoration sequences for a human to authorise. It is not automation and not a dashboard: the certified operator still decides and still carries the accountability. The defining characteristics are that it renders inside the existing HMI, states its evidence, and can decline to answer.

Does operator assist change who is accountable for a switching decision?

No, and any design that blurs it is a governance defect. Control-room operators are certified professionals — through NERC System Operator Certification in North America, through the authorised-person regime under the safety rules in Great Britain — and accountability follows the person who authorises the action. Assist changes what that person can see and how quickly they can act. Practically, this means advisory wording on proposals, unchanged accountability text in the operating procedure, and a decision log that records the operator's stated basis rather than only the system's recommendation.

How many alarms is too many for one operator?

EEMUA 191, quoted in the UK Health and Safety Executive's guidance on better alarm handling, sets a long-term average of no more than one alarm every ten minutes per operator in normal operation, and no more than ten alarms displayed in the first ten minutes following a major upset. Above roughly ten per ten minutes the alarm system has stopped functioning as an alarm system: operators silence and scroll rather than respond. These are process-industry benchmarks, and they transfer to electricity desks because the constraint they describe is the human, not the plant.

Do we have to rationalise alarms before adding AI?

Yes, and the sequence is not a preference. A correlation model trained on chattering points learns that they are informative; a cause ranker built on alarms with no defined response ranks candidates nobody can act on; and suppression rules that were never rationalised become invisible policy that no operator can interrogate at three in the morning. Rationalisation also happens to be the cheapest capability on the ladder — it releases a large share of the total value with no model at all, and it is what makes later AI spend defensible.

Why can't the assist live on a second monitor?

Because during the minutes when it is worth most, the operator's attention is committed to the alarm summary and the single-line diagram they are accountable for. A second monitor competes with an entrenched scanning pattern at exactly the moment that pattern is most rigid, so the suggestion goes unread and the programme is reported upward as cultural resistance. Separate applications are legitimate for engineers, model monitoring and post-event review. They are not legitimate for the live decision.

How do you stop operators over-trusting the assist?

Make checking cheaper than believing. Over-trust is what rational people do when a system is usually right and verifying it is expensive, so the fix is design rather than exhortation: provenance, confidence and data age one keystroke from the suggestion, on the same screen. Add explicit abstention so the system's silence carries information, and rehearse degraded and wrong-assist scenarios in the simulator — an operator who has seen it fail in training will not be surprised when it fails on shift.

What if operators ignore the assist entirely?

Treat it as an integration defect, not a people problem. Read the acceptance and view rates per suggestion type at desk level and open the override reason codes first: the usual causes are that the output arrives after the operator has already formed a hypothesis, that it renders somewhere they do not look during an event, or that its evidence cannot be reached quickly. Improving model accuracy in this situation almost never changes behaviour, because accuracy was not the binding constraint.

How do you measure operator outcomes without measuring the operator?

Aggregate everything to desk, shift type, event or suggestion type, and never to a named individual. Useful measures include alarms presented per flood after grouping, time to first correct action from the historian, suggestion acceptance and override reason distributions by type, assist availability, and CML/CI or SAIDI/SAIFI on covered feeders against a comparison group. Write the prohibition on individual-level reporting into the measurement plan, state a retention period, and agree it with the operators' representatives before any logging begins.

What should the assist do when its data is bad or it is unavailable?

Say so, in the place the suggestion would have appeared. Silent absence is the dangerous failure because operators calibrate to a system that is usually present and read an empty space as 'nothing to report'. The pattern to copy is the alarm process monitor NERC requires in IRO-018: a heartbeat that notifies the operators — not only the platform team — when the processor has failed, plus a visible degraded-mode banner, a defined fallback and a drill that proves both work.

How does operator assist interact with regulatory obligations?

It touches three areas. Training: new decision aids fall inside systematic-approach-to-training obligations for system operators, of which NERC's PER-005-2 is the explicit example, including simulator-based emergency operations training where reliability limits apply. Monitoring: real-time monitoring standards require you to be told when alarm processing fails. And classification: the EU's AI Act framework treats AI used in managing and operating critical infrastructure as high-risk, with human oversight, logging and transparency obligations that map closely to the decision log described here.

What should the assist contribute to shift handover?

A draft, never a published summary. Because the system has already logged open abnormalities, shelved alarms with their expiry times, active suggestions and recent overrides, it can assemble an accurate handover pack in seconds — which is valuable, because the HSE treats handover as a distinct human-factors risk and incidents cluster around it. The outgoing operator edits and signs it. What handover actually transfers is intent, and no model has access to intent; the machine's contribution is completeness, not judgement.

When is supervised autonomy appropriate, and when is it not?

It is appropriate for a short, enumerated set of bounded, reversible actions on a defined class of plant, where the evidence exists to state an operating envelope and where every execution can be annunciated to the desk with alarm-grade prominence and stopped by a tested inhibit. Automated fault isolation and restoration on an equipped feeder class is the canonical example. It is not appropriate for novel topologies, actions whose consequences are hard to reverse, or any case where the envelope has been extended by analogy from evidence gathered on a different class of plant.

About the author

Atomic Loops Engineering

Industrial AI practice

Atomic Loops builds production AI systems for energy, manufacturing and logistics operators — forecasting, event correlation, vision inspection and operator decision support running against live operational data, integrated into the SCADA, EMS and ADMS layer rather than delivered as a separate dashboard.

  • · Decision-support systems delivered into existing operator consoles, not alongside them
  • · Alarm rationalisation and event-correlation work run jointly with control-room staff
  • · Integration-first delivery: provenance, override capture, heartbeat monitoring, rollback
  • · 30 cited sources on this page

Sources

  1. International Society of AutomationISA18 — alarm systems standards committee (opens in a new tab)
  2. International Society of AutomationANSI/ISA-18.2 — Management of Alarm Systems for the Process Industries (opens in a new tab)
  3. EEMUAEEMUA — engineering guidance and publications (opens in a new tab)
  4. EEMUAEEMUA publications (including Publication 191, Alarm Systems) (opens in a new tab)
  5. Health and Safety ExecutiveBetter alarm handling (CHIS6) (opens in a new tab)
  6. Health and Safety ExecutiveAlarm management — human factors topic (opens in a new tab)
  7. Health and Safety ExecutiveHuman factors in control rooms (opens in a new tab)
  8. Health and Safety ExecutiveHuman-computer interaction in safety-critical settings (opens in a new tab)
  9. Health and Safety ExecutiveShift handover (opens in a new tab)
  10. Health and Safety ExecutiveCompetence and training (opens in a new tab)
  11. Health and Safety ExecutiveOperator workload (opens in a new tab)
  12. Health and Safety ExecutiveHuman factors and ergonomics (opens in a new tab)
  13. NERCSystem Operator Certification (opens in a new tab)
  14. NERCPER-005-2 — Operations Personnel Training (opens in a new tab)
  15. NERCIRO-018-1(i) — Reliability Coordinator Real-time Reliability Monitoring and Analysis Capabilities (opens in a new tab)
  16. NERCReliability standards (opens in a new tab)
  17. North American Electric Reliability CorporationNERC (opens in a new tab)
  18. International Energy AgencyEnergy and AI (opens in a new tab)
  19. International Energy AgencyEnergy and AI — executive summary (opens in a new tab)
  20. EPRIResearch programmes (opens in a new tab)
  21. National Energy System Operator (NESO)First stages of the Open Balancing Platform go live (opens in a new tab)
  22. National Energy System Operator (NESO)News and updates (opens in a new tab)
  23. Duke EnergySelf-healing technology during Hurricanes Helene and Milton (opens in a new tab)
  24. Duke EnergySelf-healing technology (opens in a new tab)
  25. Red EléctricaCecre — control centre of renewable energies (opens in a new tab)
  26. Red EléctricaCecoel — electricity control centres (opens in a new tab)
  27. OfgemPolicy and regulatory programmes (opens in a new tab)
  28. Office of Gas and Electricity MarketsOfgem (opens in a new tab)
  29. ENTSO-ENetwork codes (opens in a new tab)
  30. European CommissionRegulatory framework for AI (opens in a new tab)

Put the assist where the operator is already looking

We run the assessment with your control engineering and operations leads, read a month of alarm history and override logs, and leave you with a costed 90-day plan for one desk — alarm baseline, rationalisation tranche, console integration and the measurement plan. You keep the plan whether or not we build it.

Published · Last updated

Benchmark request

Tell us where to send it

Benchmark for this page

Used once, to send this benchmark and follow it up personally. No newsletter, no automated sequences.