Bridging the Gap Between Architecture and AI
The scariest failure in an AI system is the one where everything looks fine.
In 2025, a major professional services firm had to refund part of a 440,000 dollar government contract after a report it delivered to an Australian government department turned out to contain fabricated academic citations and a quote attributed to a federal court judge that the judge had never said. The firm admitted that parts of the report had been drafted with generative AI and issued a corrected version.[1][2]
That case is not the first of its kind, and it will not be the last. Back in 2022, Air Canada’s customer service chatbot confidently told a grieving passenger that he could claim a bereavement discount retroactively after his flight. The policy said no such thing. When he tried to claim the refund he had been promised, the airline refused, and the case ended up in front of the British Columbia Civil Resolution Tribunal. Air Canada’s defence was that the chatbot was “a separate legal entity” responsible for its own actions. The tribunal called that submission “remarkable” and ordered the airline to pay up.[3][4]
The details differ, a drafting workflow in one case and a live chatbot in the other, but the shape is the same. Dashboard green. Response times healthy. Fluent, well punctuated, entirely plausible output. And entirely wrong. Not wildly, obviously wrong, the sort of wrong you can catch with a schema check or a unit test. Wrong in a way that reads as true, and in both cases nobody noticed until a diligent outsider did.
That is the failure mode I keep coming back to. I have not lived through it in production. I have read the postmortems, watched the talks, followed the incidents as they get written up. It is what I am designing around before I put anything AI driven in front of a real customer. Every failure mode I have spent twenty years learning to design around assumes that a broken system looks broken. Timeouts. Five hundreds. Queues backing up. Certificates expiring at two in the morning. AI has introduced a new category, polite failure, and none of our existing reflexes are tuned for it.
Where the Opportunity Actually Sits
Before going further, it is worth saying out loud that I am genuinely excited about what AI makes possible right now. There is a wave of new products, new features, new modules being built on top of these models that would have been impossible or uneconomic two years ago. Automated triage systems. Document understanding at scale. Conversational interfaces that actually work. Retrieval layers over corporate knowledge that finally do the thing we have been promising for a decade. This is the most interesting time to be building software that I can remember, and the organisations that move decisively here will pull ahead of the ones that do not.
The reason I am writing about polite failure, dependency discipline and data gravity is not because I want to slow any of that down. It is the opposite. The companies that ship AI powered products successfully will be the ones whose architects put the right scaffolding in place early, so the product can move fast without the failure modes catching up with it. The discipline is what unlocks the speed, not what brakes it. Everything that follows is about how to be the team that still has momentum in eighteen months, not the team that has spent those eighteen months firefighting a system that confidently hallucinates at customers.
Polite Failure Is the Thing
Classical observability assumes success looks like a two hundred and a sensible response time. If you want to spot an AI component misbehaving, you have to be looking at the content, not just the envelope. That means evals running against a held out set. It means sampling live traffic and grading it, either with humans or with another model. It means treating prompt and model changes the same way you treat database migrations, with a rollback plan and a canary.
None of this is glamorous. None of it will make it onto a roadmap slide. But if you cannot tell me how you would know the model had quietly got worse overnight, you do not have a production system, you have a demo with good lighting.
The architectural move is to build a feedback loop around the model that is every bit as robust as the feedback loop around your deployments. Output quality becomes a first class signal, as important as latency and error rate, and it sits on the same dashboard. That is the single biggest change in mindset I would ask any team adopting AI to make.
AI Is a Dependency, Not a Feature
What I find myself saying, over and over, is that a large language model is not a feature you add to a product. It is a dependency you take on. It has an owner who is not you, a release cycle you do not control, a pricing model that will change, and a failure mode you are still learning to recognise. Treating it as a feature is like saying your payments provider is a feature. No, it is a thing you depend on, and everything about how you design around it should reflect that.
Once you frame it that way, a lot of decisions get easier. You ask the questions you would ask of any critical third party. What happens when it is slow. What happens when it is wrong. What happens when the bill arrives. What is our exit plan. These are not exciting questions, but they are the ones that separate a proof of concept from something you can put your name on.
Architecturally this looks a lot like the hexagonal patterns we have been drawing for years. The model sits behind a port like any other external capability. Validation, retries, schema enforcement, fallbacks and rate limiting all live on your side of the line. I wrote recently about circuit breakers in a slightly different context, and they are just as relevant here. You want the dependency wrapped in something that can notice it is misbehaving and route around it until it recovers.
Concretely, that means asking a question most teams skip. When the model is down, what does the user see? If the answer is a spinner and an apology, you have a brittle product. If the answer is a simpler, rule based response that is seventy percent as good and always available, you have a robust one. That gap, between “the AI is our product” and “the AI is a component of our product”, is where most of the architecture work actually happens. If your product cannot function in a degraded but useful way when the model is unavailable, you have put too much of it inside something you do not own.
Data Gravity Will Decide Who Wins
There is a phrase that keeps coming up in conversations with other architects, and it is data gravity. The idea, which is not new, is that data attracts the applications that need it, because moving data is expensive and risky. In an AI context this matters enormously, because the quality of what the model does for you is almost entirely a function of the quality and relevance of the data you bring to it.
The pattern I keep seeing is that data readiness is doing more work than model choice. The teams getting real value know where their customer records live. They have invested in some form of retrieval layer. They have thought about access control at the data level, not just the app level. The model is almost the easy bit. The hard bit is the plumbing that feeds it, and that plumbing is architecture work.
This is the conversation I would want to have with a leadership team before the AI strategy one. Not instead of, before. What is our data strategy? Because the model you pick today will almost certainly not be the model you are running in eighteen months. Look at the release cadence of the major providers and that is not a prediction, it is arithmetic. The data estate you build, on the other hand, will still be paying dividends long after the current crop of providers have merged, pivoted, or been replaced twice over. Bet on the thing with gravity.
The Work, Honestly
None of this is new in spirit. Wrap your dependencies. Watch your outputs. Invest in your data. These are the same instincts good architects have always had, applied to a technology that punishes the old reflexes in new ways.
What has changed is the cost of getting it wrong. Polite failure scales the way AI scales. A broken service loses you a transaction. A quietly drifting model loses you trust across every interaction it touches, and you do not find out until a customer screenshots it. That is not a tooling problem. It is an architecture problem.
The value of AI in most organisations will come from two places at once. The flagship products and modules that get announced to the market, and the long tail of smaller tasks that were previously too expensive to automate. Classification, summarisation, drafting, extraction, triage. The first category gets the headlines. The second quietly transforms how the business runs. The architects whose organisations ship both successfully will be the ones who did the unglamorous work. Took polite failure seriously. Treated the model as a dependency. Invested in the data before the model.
The models will keep getting better. The opportunities will keep multiplying. The job, for those of us designing these systems, is to make sure the scaffolding is ready for what we are going to build on top of it.
- Deloitte to pay money back to Albanese government after using AI in $440,000 report — The Guardian, October 2025
- Deloitte AI debacle seen as wake-up call for corporate finance — CFO Dive, October 2025
- Air Canada found liable for chatbot’s bad advice on bereavement rates — CBC News, February 2024
- Moffatt v. Air Canada, 2024 BCCRT 149 — British Columbia Civil Resolution Tribunal ruling