Who gets to keep the hard parts
A conversation with Jason B. Perkins, picking up where his review of *Deep Utopia* left off
An experienced endoscopist spends years learning to see. The skill is not in holding the camera. It is in the eye: the trained attention that catches a flat, easily missed growth at the edge of the screen, the kind that turns into bowel cancer if it is not found and cut out. That attention is built by doing the looking, thousands of times, with no one and nothing pointing the way.
Between 2021 and 2022, four endoscopy centres in Poland brought in an AI tool that draws a box around suspect growths as the camera moves. It works. Switch it on and detection goes up. But researchers also watched what happened to the same doctors when the tool was switched off again. In the months after routine AI use came in, their unaided detection rate fell from 28.4 per cent to 22.4 per cent — a fifth of their skill gone, on the procedures where they were once again looking alone. The Lancet Gastroenterology & Hepatology published the finding in August 2025. The doctors had each performed more than two thousand colonoscopies. They were not novices losing a skill they had half-learned. They were experts, and the looking had quietly stopped being theirs.
No one decided that those doctors should get worse at finding cancer. Each centre adopted the tool for an honest reason: on the day, with the box on the screen, more growths are caught. The erosion was the by-product. It arrived without a meeting and without a vote, and the act that had kept the eye sharp — the looking itself — was the very thing the tool took over.
This piece grew out of a conversation. A few weeks ago Jason reviewed Nick Bostrom’s Deep Utopia for his Book Club.
I left a comment that disagreed with the end of his argument rather than the start of it. We have been chatting it over since. What follows is both sides, written honestly and left unresolved on purpose.
We agree on Bostrom’s distinction. Some of what we do has instrumental value: we do it to reach a result. Some has intrinsic value: the doing is the point. As machines deliver the result faster, the instrumental reasons for human effort weaken, and what survives is the effort we would still choose when we no longer have to. The colonoscopy study makes me want to name a third category the distinction misses. Call it formative difficulty: effort whose result we want, and whose doing builds or sustains the capability to get that result at all. Jason reads Deep Utopia as a reason to protect that kind of difficulty on purpose. I think he is right, and I think the harder question is who gets to decide.
Jason
This is exactly why we need to stop treating AI integration as a procurement checklist and start treating it as a systems architecture challenge. If we optimise every system for friction-free convenience, we mistake efficiency for flourishing. As an architect, I read that as a classic design flaw: we optimise for the transaction while destroying the capability. We would not accept an architecture that creates security holes or data rot, and we should not accept one that causes human cognitive rot. The work is to move past “automate everything possible” toward an intentional strategy of cognitive preservation — to define which human capabilities are non-negotiable before we draw the boundary of the machine.
Three reasons the “automate everything” paradigm collapses under its own weight. The first is business continuity. Complete automation introduces catastrophic fragility. If a system needs zero human capability to run, then the day an edge case breaks the model, or the infrastructure goes dark, you do not have a temporary outage. You have total systemic failure, because the human recovery mechanism has been deskilled out of existence. Formative difficulty is business continuity insurance.
The second is the joy of creation. Bostrom reminds us that human satisfaction is tied to the process of bringing something into being. Automate the whole path from intent to output and you remove the friction that makes achievement mean anything — you are not a creator any more, you are a critic approving a menu. It also misjudges how people assign value. We do not only value outputs; we value effort. There is an intrinsic premium we place on things made by human hands and minds, a recognition of shared struggle. Erase the human trace from the work and you risk erasing the thing that makes the end product matter to anyone else.
The third is evolutionary innovation. Real innovation does not come from executing a clean process; it comes from stumbling through a messy one — the old explore-versus-exploit problem. When people work the hard parts of a problem, they notice the anomalies, the anomalies spark insight, and insight drives the next step. Automate the struggle away and you risk freezing the system in its current state for good.
That is the architecture answer, and it holds. But the colonoscopy centres were not careless. They adopted the tool for a real, same-day gain in detection, and the erosion still happened. So the question I could not let go of in Jason’s comments is who decides which difficulty is worth keeping, and on what grounds. The decision is already being made, almost always by default. No one in those rooms asked whether the unaided looking was instrumental or formative, because that is not a question a business case is built to ask. The answer arrived as a tool, and a capability that took two thousand procedures to build began to fade in months.
Bostrom says the scarce goods of an abundant world will be meaning, agency, and the texture of things done by hand. That sounds like liberation. Look again and it can describe a market, because someone always sets the price of a scarce good, and someone is always priced out. We have watched smaller versions of this. The handmade thing does not vanish when the machine arrives; it moves upmarket and becomes how the wealthy signal rather than how ordinary people live. There is no law that protects formative effort from the same gravity. The risk is not that the hard parts disappear. It is that they are kept by whoever can still afford to do things the slow way, and lost by everyone measured on speed.
Jason
You are pointing at the gravity well that swallows most enterprise technology strategy: the default assumption that efficiency is an absolute good. But I do not concede that formative difficulty has to become a luxury or a productivity penalty. It does demand a real shift in how we govern and architect systems. In enterprise architecture we use the idea of a mesh to balance competing forces, and we need a comparable pattern for human-AI interaction — one that rejects the binary between the unaided human and the autopilot machine. Build systems that treat human capability as a core asset to be maintained and valued, rather than a labour cost to be depreciated, and you change the economic calculus. The institution does not have to choose between today’s productivity and tomorrow’s capability. But to get there, the boardroom has to stop seeing AI as a way to replace the human eye and start seeing it as a way to sharpen it.
There is a harder question underneath this one, and neither of us should pretend it is settled. The world is messy, and there are challenges well beyond today’s struggles. Does removing today’s difficulty open room for higher-order meaning? Next-generation AI will not simply erase work; it will create domains of work and capability we cannot yet define. The frontier always moves. Zack Kass poses the question I keep coming back to: if you could automate everything in your life, where would you stop?
If we are going to defend any difficulty, the test has to be sharper than nostalgia, because most appeals to the value of struggle are sentimentality about how things used to be done. Plenty of difficulty is worth removing. No one should defend long division by hand, or a three-week wait for a letter, or a disease we can now cure. The difficulty worth keeping is the narrow kind that builds or sustains something in the person doing it: skill, judgement, identity, or a relationship that would not otherwise exist. The endoscopists’ looking sustained an eye for cancer. The tool kept the detection and let the eye go, on the quiet assumption that the eye was a by-product rather than the point.
Jason reaches the same line from the other direction, and calls it virtuous and vicious friction. Vicious friction only diminishes us: the pointless meeting, the layer of bureaucracy, the months spent waiting for access. Virtuous friction defines us: the apprenticeship that builds judgement, the demanding conversation that builds trust, the hard way that leads to mastery. He started from the architecture and I started from the person, and we arrived at the same cut. Not effort against ease. Formative difficulty against incidental difficulty. The trouble is that the two are indistinguishable on a dashboard, where both show up only as time the machine could save, and the people best placed to tell them apart — the doctors who can feel their own eye dulling — are rarely the ones holding the budget or writing the rollout plan.
None of this is an argument against the tools. Those doctors should have the AI; on the day, it finds more cancers, and that is not a small thing to set aside. The argument is that someone has to be watching the difference between the difficulty that was keeping a skill alive and the difficulty that was only costing time, and at the moment almost no one is. The Polish researchers were careful: their study was observational, and they could not yet say what happens to doctors who train with the box from the start and never build the unaided eye at all. That is the more troubling question, and it is still open.
We will not lose the hard parts in a single decision anyone could be held to. We will lose them one defensible efficiency at a time, each removal sensible on its own, until a generation arrives that can check the machine’s work but could never have done it alone. If meaning, judgement, and skill really are the scarce goods of the world we are building, the choice in front of every organisation is not whether to keep some difficulty. It is who gets to decide which difficulty, and whether the person it would have formed is ever in the room when the decision is made.
Over to you. Before you automate the next hard thing, Jason offers five questions worth holding it against:
Agency — does it strengthen human control, or quietly remove it?
Connection — does it deepen trust and human bonds?
Judgement — does it sharpen our ability to decide, or outsource it?
Opportunity — does it widen what people can collectively become?
Safety — can the outcome be audited or reversed?
So we will put the same question to both our readerships and answer it in the thread: which difficulty in your own work is keeping a skill alive, and which is just slow?
Sources: Budzyń K, Romańczyk M, Mori Y, et al. “Endoscopist deskilling risk after exposure to artificial intelligence in colonoscopy: a multicentre, observational study.” The Lancet Gastroenterology & Hepatology, August 2025 (DOI: 10.1016/S2468-1253(25)00133-5). Figures (28.4% to 22.4% unaided adenoma detection; 19 endoscopists; four Polish centres; 1,443 non-AI colonoscopies) per the study and The Lancet press release, 12 August 2025. The “automation boundary” question is drawn from Zack Kass, as cited by Jason; it is a paraphrase, not a verbatim quotation.
You’re reading The Next Evolution by Neil Catton, articles that explore the human world and the intersection of technology, they try and ask difficult questions - not to scare - but to inform. If someone forwarded this to you, you can subscribe free at neilcatton.substack.com.
Neil Catton is the author of The Next Evolution, The Cognitive Crucible and The Shadow System - available on Amazon, and writes at the intersection of technology, ethics, and human purpose.