In a recent interview, Steve Hasker, chief executive of Thomson Reuters, said that in fiduciary professions like law, tax, audit, and compliance, what matters is not just speed but whether the output is authoritative, traceable, and accountable to professional standards. The same applies to insurance claims.
For a while, much of the conversation around AI focused on efficiency. Could it help insurers process claims faster? Could it reduce manual work? Could it improve consistency? Those are still important questions. But they are no longer the only ones that matter. The harder question now is whether the decisions AI helps shape can be understood, defended, and trusted.
This matters even more in claims, where a decision is rarely truly final when it is first made. It may be questioned by the policyholder, examined by regulators, revisited in an audit, or challenged in court. Under that kind of scrutiny, a system cannot just give an answer. It also needs to make clear how it reached that conclusion.
This is why explainability has moved from a mere technical concern to a business requirement. Explainability now sits at the center of governance, compliance, and customer trust. The broader market is also moving beyond initial excitement about what AI can do in theory and toward a more practical question: What does it take to use it responsibly in real operations? A recent Gartner report points to the same trend, highlighting governance, data readiness, and operational discipline as the factors that will separate early excitement from lasting value.
In claims, explainability is where that shift becomes real.
AI Does Not Change the Insurer's Responsibility
AI may be changing how work gets done, but it does not change the insurer's underlying obligations. Claims laws, consumer protections, and standards for fair treatment still apply, whether a decision is made by a person, a model, or a third-party vendor. If a claim is denied, delayed, escalated, or flagged for possible fraud, the carrier is still responsible for being able to explain why.
This is becoming even more important as regulators pay closer attention to how AI is used in insurance. In Europe, the EU AI Act raises the bar for transparency, explainability, and human oversight in high-risk uses of AI. In the United States, the regulatory picture is less uniform, but the direction is similar: insurers are increasingly expected to show that AI-supported decisions are fair, accountable, and subject to oversight.
For claims leaders, the takeaway is simple. The issue is no longer whether AI is allowed in claims. The issue is whether the carrier can stand behind the decisions it helps produce.
The Real Divide Is Not Who Uses AI
Across property and casualty insurance, the appeal of AI is easy to understand. Carriers want to improve consistency, reduce handling costs, and help claims professionals manage growing complexity. Used prudently, AI can support all of these goals.
But the real divide in claims is no longer between insurers that use AI and insurers that do not. It is between systems that produce outputs and systems that produce decisions the business can actually explain and defend. This difference matters because a decision made today can come back months or years later in a complaint, an audit, or a lawsuit. A recommendation that looks efficient in the moment can become a serious problem later if no one can clearly explain why it was made.
This is where black-box neural network systems begin to create friction. Models built for speed and prediction may perform well in testing, but if their reasoning cannot be understood in plain terms, every downstream review becomes harder. Claims teams are left trying to reconstruct logic after the fact from technical outputs or vendor explanations. By then the problem is no longer just technological; it becomes operational, legal, and reputational.
Why Explainability Cannot Be Added Later
One of the most common misperceptions among carriers is that explainability can be dealt with later, after a model is already in production and the business value has been proven. In practice, that is far more costly than it sounds.
Once a claims process is built around systems that do not make their reasoning easy to follow, real transparency is difficult to add later. You can layer on reports, write summaries with LLMs, or use tools such as SHAP values to suggest which factors may have shaped the outcome, but these are still only approximations. They are not the same as being able to see the path to the decision from the beginning.
In the real world, repair costs change, fraud tactics evolve, and what counts as normal in one region may not hold in another. These shifts can easily affect a black-box AI system, and the problem is often hard to see until real harm has already been done. By then, it may appear as a rise in customer complaints or as questions from a regulator about why similar claims are being handled differently.
The Gartner report makes much the same point in broader terms. The companies most likely to get lasting value from AI are the ones that build the right structure around it: good data, clear accountability, and controls that remain dependable over time. In claims, explainability is part of that structure. It is not something you can add later. It has to be there from the beginning.
The Problem Often Starts with the Data
Explainability is often talked about as if it lives only inside the model. In reality, it depends just as much on the data feeding the system.
A claims system cannot produce trustworthy reasoning if the underlying data is incomplete, poorly structured, weakly governed, or disconnected from the market where it is being used. If the inputs are flawed, the explanation may sound polished while still hiding the real problem. A carrier may think it has a well-controlled system, but if claim data varies widely across geographies, repair networks, documentation practices, or policy types, the system can still produce unstable or unfair outcomes.
Explainability helps bring those problems to the surface. It gives the business a way to spot when the model is leaning too heavily on the wrong signals, when patterns do not fit local conditions, or when assumptions that made sense in one setting are being carried into another where they no longer belong.
This was illustrated in an auto claims fraud model deployed in a Middle Eastern country. The legacy model consistently flagged accidents occurring after midnight on weekends as high risk. That logic had been inherited from North American and European training data, where late-night weekend accidents often correlate with alcohol-impaired driving. But in the market where the model was being used, alcohol consumption was prohibited and families commonly stayed out late on weekends. The signal was not just weak; it was systematically wrong. Because the reasoning behind the model's behavior was not visible in a usable way, the problem went unnoticed until it showed up in regulatory scrutiny and customer dissatisfaction.
What Explainability Should Look Like in Practice
The first question claims leaders should ask about any system influencing claim outcomes is not just whether it is accurate. It is whether the people responsible for the claim can understand the reasoning well enough to use it responsibly.
If a claim is delayed, escalated, denied, or flagged for possible fraud, the adjuster should be able to see the factors driving that result in language that makes sense in the context of the claim. That could include missing documents, timing issues, policy conditions, behavioral patterns, or other relevant signals. The point is not that every decision becomes simple. The point is that the reasoning should be visible enough for the business to evaluate it.
Just as importantly, the adjuster should be able to challenge the recommendation when it does not fit the facts. Human oversight only means something if the person reviewing the claim can see why the system is pointing in a certain direction and can document a reasonable override when needed. Otherwise, "human in the loop" becomes little more than a formality.
Good explainability also helps the organization over time see when outcomes are drifting, when similar claims are being treated differently, or when a signal that once seemed useful is starting to distort results. In that sense, explainability is not just about answering questions after a problem appears; it is also about spotting problems early enough to do something about them.
A Practical Test for Claims Leaders
For claims executives, the best way to judge whether a system is ready is to ask a few simple questions.
Can we explain how this AI-generated recommendation was produced?
Can we trace the data and reasoning behind it without relying on a technical team to rebuild it from scratch?
Can we show that similar claims are being treated consistently?
Can we detect when the AI model's behavior starts to shift?
Can an adjuster disagree with the AI-generated recommendation and explain why in a credible way?
These are not abstract governance questions, rather practical tests of whether the AI model can survive real-world scrutiny.
The Choice Facing Claims Leaders
Claims leaders now face a much clearer choice than many realize.
The question is no longer whether AI can improve speed, support triage, or strengthen fraud detection. In many cases, it can. The real question is whether insurers are building those capabilities on foundations strong enough to hold up when the decision is reviewed, challenged, or questioned later.
That is why explainability matters so much now. It is not just a safeguard for compliance teams, but also the practical link between AI and accountability. It connects decisions to oversight by helping expose weak data, hidden bias, and changing patterns before they become bigger problems.
The wider business conversation is moving in the same direction. As the early excitement around AI gives way to a more realistic view, companies are becoming clearer about what lasting value actually requires. The real issue is no longer just what AI can do, but whether it can be trusted, governed, and used responsibly over time. In claims, explainability is where that becomes visible in everyday decisions.
