Measuring value every quarter

Why measurement decides the programme

Most companies cannot say what AI has delivered. In McKinsey’s 2026 survey, nearly nine in ten companies use AI regularly, but only 37% attribute any EBIT impact to it. Some of that gap is real. Some of it is simply unmeasured.

The companies pulling ahead measure differently. BCG’s 2026 index found that the leaders measure AI’s impact directly in their financial results, and those leaders show 2.4 times the revenue growth and 2.8 times the EBITDA growth of laggards. Measurement is how a programme earns its next budget, and how leadership knows where to put it.

What to stop reporting

Many AI reports lead with activity: number of licences, number of users, prompts sent, pilots started. These numbers are easy to collect and say little about value. A company can have every employee using an assistant and no change in its P&L.

Activity numbers still have a place, lower down, as early signals of adoption. They should never be the headline.

Four numbers per agent

For each live agent, report four numbers, each against its baseline or target from the business case in Chapter 6.

1. Value. The change in the P&L line the agent was built to move: cost per item, spend, hours redeployed, revenue, errors avoided. In money where possible.
2. Quality. Share of items handled without change, override rate, and results on the standing test set from Chapter 7.
3. Cost to run. Model usage, infrastructure and review time per item, and the monthly total.
4. Risk. Incidents, guardrail triggers, complaints, and the status of the controls required for the agent’s tier in Chapter 8.

Show each as a trend over the last four quarters. A single quarter rarely tells the story.

The UK water utility from Chapter 2 is a good example of numbers that hold up. The communications assistant scores more than 3,000 communications a month, saves the communications team around 18 hours a week, and produced zero non-compliant language flags in regulatory bulletins during the pilot. Value, volume and risk, each in a form leadership can read in seconds.

The portfolio view

Above the individual agents, the board needs one view of the programme:

One page for the portfolio. One page per agent in the appendix.

Reading the trends

The numbers tell a story when read together.

• Override rate falling, quality stable. The agent is earning trust. It may be ready to move to the next mode, with fewer reviews.
• Override rate rising. Something has changed: the data, the model or the process. Look before it shows up in value.
• Value flat, cost rising. Volume is growing in cases the agent handles poorly, or review time is too high. Revisit the scope.
• Value ahead of plan. Consider whether the same agent can serve another team or site, at a small extra cost.

Twistag tracks its own engineering agents the same way. Pulse reports quality, security, cost and return, activity, people, adoption and operations, and leadership watches the override rate for each check, by team, over time. The trend matters more than any single week.

The quarterly rhythm

A simple rhythm keeps measurement useful:

• Monthly: each agent owner reviews their four numbers with the operator and engineer.
• Quarterly: the leadership team reviews the portfolio page and makes decisions on each agent and on the pipeline.
• Twice a year: the board sees the portfolio, the governance status and the plan for the next six months.
• Yearly: the short list from Chapter 1 is rebuilt from scratch, with what the company has learned.

Every quarterly review ends with decisions. Scale an agent to more teams or volume. Change its scope or mode. Stop it, and release the budget. Start the next use case from the short list.

Making the numbers credible

Three practices make the numbers hold up when finance or the board looks closely:

• Finance owns the value logic. The method for turning hours, errors or spend into money is agreed with finance before launch, and does not change quarter to quarter.
• Baselines stay fixed. The before number is recorded once, during the proof, and every later number is compared with it.
• The data comes from the systems. Value and cost are read from the same traces, logs and financial systems every quarter, without manual estimates.

Common mistakes

• Reporting activity as value. Licences and logins at the top of the report.
• Moving baselines. The before number is revised each quarter, and the improvement becomes impossible to verify.
• Only success stories. Agents behind plan are left out, and the board loses trust in the whole report.
• No decisions. The review ends with a discussion and no change to the portfolio.
• Measuring once. Value is checked at launch and never again, while quality and cost drift.

What to do this month

1. Agree with finance how value is calculated for each live agent.
2. Set up the four numbers per agent, read from systems where possible.
3. Build the one-page portfolio view, and use it at the next leadership meeting.
4. End that meeting with a decision on every agent: scale, change or stop.

Questions for the leadership team

1. What value did AI deliver last quarter, in money, and does finance agree with the number?
2. Which agents are behind their business case, and why?
3. What does each agent cost per item today, compared with three months ago?
4. Which agents are ready for more room, and which need closer review?
5. Which use case on the short list should start next quarter?