It Cannot Automate Trust
On automated threats, static defences, and the one gap in the criminal economy's attack chain that almost no one is investing in.
Morgan had been reading about all of it. The scams. The data breaches. The criminal economy with its support desks and refund policies. The state attacks that nobody calls attacks. The encryption that will eventually stop working. Morgan read the articles, followed the links, understood more than most people do, and then sat back and asked the only question that actually matters: so what am I supposed to do?
Not a list. Not a training course. An honest answer.
The answer most people receive, if they receive one at all, is a set of instructions. Use a password manager. Enable two-factor authentication. Don’t click links in emails you weren’t expecting. These are not wrong.
They are also not sufficient, and everyone who gives them knows they are not sufficient. The threat is not a password problem. It is a civilisational one.
And yet the only thing most people are offered is a checklist.
Here is what the checklist misses. The person who clicks a link in a phishing email is not, in the main, careless. They are tired, or hurried, or the email arrived at the exact moment something else was happening. They are human.
The criminal economy knows this, which is why it has spent twenty years optimising against human behaviour rather than human defences. It has automated the finding, the targeting, the delivery, and the follow-through. The checklist has not kept pace because a checklist cannot keep pace. It is a static defence against a dynamic threat.
The thing that can keep pace is also human. But it does not come in a list.
The threat is automated. The defence, ultimately, has to be human.
Consider what actually happens in the moments before most digital attacks succeed. In many documented cases, someone had seen the warning signs, a colleague mentioned a strange email, a neighbour said they’d had an odd phone call, something felt slightly off. The warning existed. It just was not passed on in time, or in the right direction, or to someone who knew what to do with it.
The criminal economy has automated everything it can on its side of the transaction. It can now clone a voice, fake a face, and mimic an email from someone you know. What it cannot yet do reliably is insert itself into the informal network, the passing warning, the sideways mention, the “I heard something odd” between two people who already have a reason to listen to each other. That gap is narrowing. But it has not yet closed.
That moment is the only defence that genuinely scales faster than the threat. Not because it is flawless. Because it is alive.
Think about what a security operations centre can and cannot do. It can monitor traffic patterns, flag anomalies, and correlate signals across a network. What it cannot do is reach the right person at the right moment with a warning that lands. A warning delivered between two people who know each other does exactly that specifically, in the moment, to this person about this threat.
It also carries something no algorithm can carry, which is trust. Not institutional trust or brand trust, the trust between two people who have a shared history and a reason to take each other seriously. That is what makes the warning move rather than be dismissed.
And it adapts in a way that no firewall or detection model can match. It responds to the particular situation of a particular person — their habits, their vulnerabilities, their routines — in real time. The fraud detection system works from statistical averages. The person who knows you works from you.
If the defence is human
The argument above isn’t an instruction. It is an observation about where the gap remains that the automated threat hasn’t yet closed. The questions below are for the person reading this who already understands the technical dimension and hasn’t yet asked whether the human one is their responsibility too.
Is there something you know about digital risk that the people around you don’t and have you told them?
What would it take to build a culture where passing on security awareness is as normal as passing on a recipe?
If the most powerful defence is human and social rather than technical why is almost all the investment going into technology?
The arguments in this article draw on Resilience by Design and The Fortress Within in Part VI of The Shadow System by Neil Catton — an examination of the digital threat as a single self-reinforcing system that has outpaced every institution designed to govern it.
Authors Note:
Morgan is a fictional character. Their story is drawn from a combination of professional observation and personal proximity to real events. The experiences described are real. The person is not.
You’re reading The Next Evolution by Neil Catton, articles that explore the human world and the intersection of technology, they try and ask difficult questions - not to scare - but to inform. If someone forwarded this to you, you can subscribe free at neilcatton.substack.com.
Neil Catton is the author of The Next Evolution, The Cognitive Crucible and The Shadow System - available on Amazon, and writes at the intersection of technology, ethics, and human purpose.