Domain: Finance & Economics | Arc: Human Cost | Theme: A decision with no name is a decision with no owner
Declined.
That was all it said. One word, in a notification that arrived on a Tuesday morning while Jordan was getting ready for work. No explanation beneath it. No number to call. No next step suggested. Just the word, and a timestamp, and a reference number when typed into the lender’s website led to a page that said the decision had been made and could not be reviewed online.
They had been saving for six years. The deposit was in place. The property had been found after months of searching — the right size, the right location, the right price for what the salary could support. The numbers had been run carefully, the way people do when they have been waiting a long time for something and do not want to get it wrong. By every measure that could be identified, this should have worked.
The system had concluded otherwise. It had not said why. It had not offered a route to find out. It had simply processed the application, weighed it against whatever it was weighing applications against, and returned its verdict. The conversation, such as it was, was over.
The wall
The number on the website was called. An automated menu offered four options, none of which covered the relevant situation. Option three — general enquiries — transferred to a queue with a twenty-five-minute wait. Eventually a person answered, listened, and explained that mortgage decisions were handled by a separate team who could only be contacted in writing. The address turned out to be an email address. The email was sent.
The reply arrived four working days later. It confirmed that the application had been assessed in accordance with the lender’s lending criteria. It said that for data protection reasons, specific details of the assessment could not be shared. It provided a link to the lender’s complaints procedure, which explained that complaints would be assessed within eight weeks.
Eight weeks. The property had a buyer interested. The estate agent had already been in touch.
The complaints procedure was followed. The outcome, six weeks later, was a letter confirming that the lending decision had been reviewed and had been found to be in accordance with the lender’s policies and procedures. It did not explain what those policies and procedures had found. It did not say what had triggered the decline. It said the Financial Ombudsman Service was available if dissatisfaction remained.
The property was gone by week three.
This is not a story about a corrupt system or a malicious one. The people involved were polite. The process ran as it was designed to run. That is precisely the problem. The absence of explanation was not a gap in the design — it was the design. The system was not built to explain itself. And nobody, at any point in the process, had the authority or the information to say why the decision had been made, because nobody had made it. The algorithm had.
The accountability gap
Algorithmic decision-making in high-stakes contexts is not new. Credit scoring has used statistical models for decades. What has changed is the scale, the opacity, and the speed at which these systems have been deployed, and the gap that has opened between the decisions they make and the accountability structures that should sit alongside them.
In the UK, 75% of financial services firms are already using AI, according to a Bank of England and FCA survey published in November 2024. AI underwriting is now deeply embedded across retail lending. Some lenders offer mortgage decisions within 24 hours. Decisions that once took teams of analysts weeks now happen in seconds, assessed against models whose internal logic is not visible to the person being assessed, the organisation deploying the system, or the regulator overseeing it.
The regulatory frameworks exist on paper. UK GDPR Article 22 gives individuals the right not to be subject to solely automated decisions with legal or similarly significant effects; Article 15 gives the right to receive meaningful information about the logic involved. The FCA’s Consumer Duty requires firms to deliver good outcomes for retail customers, and the Senior Managers and Certification Regime places accountability on named individuals for the conduct of their firms.
In practice, these frameworks share a structural problem: they were not designed for systems that cannot articulate their own reasoning.
The FCA’s October 2024 Treasury Committee session identified transparency and explainability in lending as a supervisory priority, the regulator has recognised the gap but has not yet closed it. No prescriptive AI-specific rules apply to mortgage lending. The February 2025 ruling from the Court of Justice of the EU clarified that providing an algorithm alone does not satisfy the explanation obligation — the explanation must relate specifically to the individual’s case. That ruling provides legal direction. It does not provide the infrastructure that would make compliance achievable.
Beneath all of that sits something more troubling. Earlier this year, academic research published in a peer-reviewed law journal concluded that the Data (Use and Access) Act 2025 has not strengthened the UK’s protections in this area. It has weakened them, introducing an ambiguous standard around ‘meaningful human involvement’ and restricting key safeguards to decisions involving special category data only. The paper described this as creating significant legal gaps at precisely the moment AI deployment in consumer lending is accelerating.
The regulatory direction of travel is moving away from the individual’s interests at the same time as the technology is moving further into their life.
Did you know?
Under UK GDPR Article 22, individuals have the right not to be subject to solely automated decisions that produce legal or similarly significant effects — which includes a mortgage application decision.
Where such a system is used, Article 15 gives individuals the right to receive meaningful information about the logic involved — not a reference to the algorithm, but a genuine explanation specific to their case. The Court of Justice of the EU confirmed this in February 2025.
In practice, legal scholars have described the UK’s current framework as creating significant gaps between the right as stated and the right as enforceable.
If you have been declined by an automated system and received no meaningful explanation, you have the right to ask for one. Whether you will receive one that is legally adequate is a different question.
What it costs
The immediate cost was the property, the timing that could not be recovered, the plan that had to be rebuilt from a different starting point. That cost is real and measurable, even if it does not appear in any dataset about algorithmic decision-making.
The deeper cost is less visible but more lasting. There is no way to improve what cannot be seen. Was the income calculation accurate? Was the employment history read correctly? Was a postcode being used as a proxy for a risk profile that bore no resemblance to the actual situation? Was the training data that shaped the model built on patterns from a decade ago, from a different economic context, from a population that has nothing in common with the person being assessed?
There is no way to know. And because there is no way to know, there is no way to prepare, improve, or respond. The opacity is permanent, not temporary.
The FCA and the Prudential Regulation Authority (PRA) have both flagged concern that AI-enabled hyper-personalisation in lending could lead to the exclusion of some groups from financial products entirely, making certain customers effectively un-mortgageable without those customers ever understanding why. Postcode has been identified as a proxy variable in mortgage algorithms. Employment status, income pattern, and the digital footprint of how an application is completed have all been shown in research to carry information the borrower did not intend to provide.
A 2025 survey by the Ada Lovelace Institute found that 59% of the UK public want clear procedures for appealing AI decisions to a human being. That 59% have not, in most cases, been given what they want. The same survey found that 72% want laws and regulation to govern AI use, an increase of ten percentage points from 2022. The demand for accountability is rising. The infrastructure is not keeping pace.
Three decades of watching organisations deploy technology for throughput and cost reduction produces one consistent observation: the people who commission these systems almost never experience them as recipients. The asymmetry between the decision-maker’s experience and the affected person’s experience is not an accident. It is a design outcome. It is the most important thing to understand about why the accountability gap persists.
What the system failed to do
The system did not assist. The person who engaged with it ended up worse off, not just in outcome but in understanding. A process that leaves someone less informed than they were before they began has not assisted them. It has processed them.
What used to sit in the relationship between lender and borrower — a conversation, a judgment, a human being with the authority to weigh context and ask questions — has been replaced by something that carries the weight of a decision but none of its accountability. There is a real case for AI in lending: human underwriters carry their own biases, and consistency in applying criteria is genuinely valuable. But consistency applied to a flawed model is consistent failure. Bias in a training dataset is not neutrality, it is bias at scale, applied without the ability to recognise or challenge it.
An algorithm trained on historical patterns cannot see an individual. It can only see the pattern an individual superficially resembles. A career change, a period of freelance work, an income structure that does not fit the standard model none of these register. The criteria are applied uniformly, which is another way of saying the assumptions are applied uniformly. This is not a UX problem that better design would fix. It is structural.
Fully human review of mortgage applications at the volume UK lenders process is not realistic, and nobody is arguing algorithms should not exist. The question is what minimum accountability infrastructure looks like, what explanation, what appeal route, what human review for edge cases, and whether that infrastructure has been built. In most cases, it has not. The algorithm arrived. The accountability infrastructure did not.
Four questions
Think about what this could actually mean as we deploy more algorithms throughout decision making systems, and what future challenges the emergence of Agentic AI will bring. But this isn’t about systems, it’s about people - it’s about you, so ask:
Has this happened to you, or to someone close to you, declined, filtered, assessed, without explanation? Not just in mortgage lending, but in any process where an automated system produced a verdict and offered no route to understand it?
If you work with or commission systems like these, do you know what happens to the person on the other side? Not in aggregate, individually. What does the experience of receiving a decision from your system feel like for the person who did not get what they needed?
If you could change one thing, a plain explanation, an appeal route that reached a person with genuine authority, a human review for declined applications, what would it be? What would it cost to build, measured honestly against what the absence of it costs the people on the receiving end?
And the hardest one: the trade-off between speed and scale on one side, and accountability on the other, is it one we consciously chose? Or is it one that happened to us, decision by decision, deployment by deployment, while nobody was asked whether this was what we wanted?
A note on Jordan
Jordan is a fictional character. Their story is drawn from a combination of professional observation and personal proximity to real events. The experiences described are real. The person is not.
You’re reading The Next Evolution by Neil Catton, articles that explore the human world and the intersection of technology, they try and ask difficult questions - not to scare - but to inform. If someone forwarded this to you, you can subscribe free at neilcatton.substack.com.
Neil Catton is the author of The Next Evolution, The Cognitive Crucible and The Shadow System - available on Amazon, and writes at the intersection of technology, ethics, and human purpose.