The Company That Measured What Mattered
When the dashboard is green and the person is not
Sam is presenting the quarterly review. The slide on screen is mostly green. Transaction volume is up. Response times are within target. Cost per interaction has fallen for the third consecutive quarter. The team has worked hard and the operational data confirms it.
Then Sam shares the second slide.
It shows the results from the human outcome survey, a question the organisation introduced fourteen months ago and now treats as a primary metric alongside the operational data. The question is this: did this make your life easier? The percentage who answered yes is lower than the dashboard would suggest. No one says anything for a moment.
The gap the dashboard couldn’t see
One finding sits at the centre of the silence. The account management flow, the one the team had spent a quarter building, the one that had a 94% completion rate in the transaction logs, was completing. It was just not helping. People were finishing the interaction and then calling the contact centre to do the same thing again, because the digital process had technically resolved their request without giving them the information they needed to act on it.
The operational data had no way to see that. Completion was completion. A completed transaction looked identical whether the person on the other side was better off or not.
What made this particular finding difficult was not what it revealed about the technology. The technology had worked as designed. What it revealed was the distance between the organisation’s design assumption and the person’s actual need, a gap that the operational metrics had been measuring around, consistently and accurately, for six months without once pointing at it.
The human outcome survey had pointed at it in its first cycle.
Two things that look like measurement
Most technology measurement is built around what is easy to measure: transaction volume, uptime, response time, cost per interaction, a satisfaction score. These are genuine and necessary. They tell you whether the system is working. They do not tell you whether the person is better off.
That distinction matters because the two things can diverge. A system can process every request correctly, within time, below budget, and within the satisfaction score threshold, while the person using it is not meaningfully helped. When system performance and human outcome are aligned, the operational data is sufficient. When they diverge, the operational data will always miss it because it was never designed to see it.
The investment Sam’s organisation made was in the measurement that sits between those two things: a regular, person-centred question asked of a consistent cohort and treated as a primary metric, not a supplementary one. The question was short, the cohort was manageable, and the review cycle was quarterly. What it required was the willingness to know the answer and to treat an uncomfortable answer as information rather than as a problem to manage.
That willingness is the harder investment. And it is harder to make than it sounds when the comfortable measurement is already available, already accurate, and already green.
What distinguished Sam’s organisation from those that have attempted something similar and retreated was not the sophistication of the method. It was three conditions applied simultaneously. The question was written from the person’s perspective, not the organisation’s, not did the interaction resolve according to our process, but did this make your life easier. It was treated as a primary metric, not a supplementary one, not a footnote to the operational dashboard but a standing item in the same governance conversation as the cost and completion data. And the findings were acted on: the gaps the measurement revealed led to specific changes in the next sprint, not to a report that sat in a folder and was referenced in the following year’s strategy document.
Remove any one of the three and the measurement produces knowledge without consequence which, over time, is indistinguishable from not measuring at all.
What the measurement asked them to do
The account management flow was redesigned. The team removed three steps, rewrote the final confirmation screen to include the specific information people had been calling to ask for, and added a direct link to the next action in the process. The completion rate held. The follow-up contact-centre calls for the same request fell by more than a third over two quarters. The human outcome score for that interaction moved above the organisational threshold.
A second finding produced a different kind of change. The notification system, the automated message that told people their request had been received, was generating confusion rather than confidence. People could not tell from the notification whether anything was being done, or when. It was technically accurate. It was not useful. The content was rewritten, not the system. Three sentences became one. The call rate for status queries dropped sharply in the following quarter.
A third finding was harder to act on. A feature the development team had invested six weeks in was being used by fewer than eight per cent of the people it had been built for. The human outcome data showed it was not making their lives easier. The feature was not removed, the minority who used it found it helpful, but it was deprioritised in the roadmap, and the equivalent capacity was allocated to a flow the measurement had identified as a real source of difficulty.
None of these changes were dramatic. What was significant was the cumulative direction: a product moving, over time, closer to the experience the people who used it actually needed, because the organisation was measuring the distance and choosing to close it.
The questions operational data cannot answer
Each of the three changes answered a different question and all three are questions that operational data cannot answer.
The account management flow had been answering the wrong one. The system was processing. The person was not better off. Those are different things, and a completion rate measures only the first. An organisation that restricts itself to that question will build technology that processes correctly and never quite serves. Sam’s organisation chose to ask the other one, did the person’s situation improve?, and to invest in knowing the answer rather than assuming it.
The notification redesign revealed something else. The notification was not wrong; it was simply not adding anything. It created work rather than removing it. An organisation that measures whether its technology adds something for the person, not whether it generates an output, will make different decisions from one that measures output alone. The redesign took a day. The measurement that revealed its necessity took fourteen months of consistent data collection to surface.
The deprioritised feature answered the hardest question: does the technology respond to the people who are actually using it, rather than the population the team imagined when they designed it? It had been built for a user group that turned out to be smaller than the design assumption. The measurement revealed the gap. What closed it was not a rebuild but a reallocation — one that the operational data had no mechanism to surface, and the human outcome data made visible in its second review cycle.
This last question is in some ways the most consequential. The first two reveal whether the technology is working for the person as designed. The third reveals whether the design itself was right. An organisation that measures human outcomes consistently will eventually discover that the system is not reaching the range of people who actually use it, because no system does, at first. What it does with that discovery is what separates an organisation that builds trust from one that builds a persuasive story about its metrics.
The measure your organisation has not made
The case Sam’s organisation makes is not that human outcome measurement is complicated. It is that it requires a decision most organisations have not made: to know the answer when the answer is not flattering. The gap that decision would close is specific. So are the conditions that keep it open.
The starting point is the primary metrics your organisation uses to assess the success of its technology. Do any of them measure whether the technology made someone’s life easier from their perspective, not the organisation’s? If not, what would the closest available measure be? And if that measure exists but is not in the primary review, that absence is worth explaining.
If you are responsible for a technology measurement approach: when did you last present a finding to leadership that was uncomfortable, that revealed a gap between system performance and the person’s experience? What the culture does with that kind of finding is the real test of whether the measurement matters.
If you could add one human outcome measure to your organisation’s technology dashboard, something that would tell you whether the technology is working for the people it serves, what would it be? The answer to that question usually already exists somewhere in the organisation. The question is whether it is being counted.
The organisations that measure what matters build more trust with the people they serve than those that optimise the metric. Is that trust worth the investment in harder measurement? If your organisation is not making that investment, the reason is worth examining because it is usually not that the measurement is impossible.
The gap between system performance and human outcome is not a technical problem. It is a measurement decision. Most organisations choose the comfortable measurement. The gap stays.
Authors note:
Sam is a fictional character. Their story is drawn from a combination of professional observation and personal proximity to real events. The experiences described are real. The person and organisation is not.
You’re reading The Next Evolution by Neil Catton, articles that explore the human world and the intersection of technology, they try and ask difficult questions - not to scare - but to inform. If someone forwarded this to you, you can subscribe free at neilcatton.substack.com.
Neil Catton is the author of The Next Evolution, The Cognitive Crucible and The Shadow System - available on Amazon, and writes at the intersection of technology, ethics, and human purpose.
This really resonates. In a world of 'easy' data, we often default to vanity metrics because they provide a false sense of certainty. True metrology, the art of it - requires embracing the tension between the quantitative and the qualitative.
High-impact living measures aren't meant to be 'comfortable' numbers; they should act as a mirror that forces us to ask better questions about our judgment and impact.