Explainable AI Is Not Enough

You cannot explain your way out of algorithmic bias. You have to intercept it before the rate exists/ 

Explainable AI

If you lead architecture, pricing or compliance in personal lines, you have probably been handed two mandates in the same quarter: get predictive pricing into production faster, and be ready to show a state examiner that it does not discriminate. The standard answer to the second mandate is explainable AI. I spent a year testing whether that answer holds up, and it does not, not because XAI is bad but because it arrives too late to do the job carriers are assigning it.

The truth is that you cannot explain your way out of algorithmic bias. You have to intercept it before the rate exists, and interception costs far less in latency than most architecture teams assume.

Where the current approach breaks

Personal lines pricing has moved from static actuarial tables to multi-layered predictive models fed by telematics and third-party data. The risk stratification is better. The governance exposure is different in kind from anything traditional model validation was built to catch: a deep neural network can construct proxies for protected classes on its own, from variables such as garaging ZIP code or credit history tenure, without a protected attribute ever appearing as an input.

Most teams answer this with post-hoc explainability. Those methods do real work: they describe how a model reached an output and make audit response tractable. But explanation is retrospective by construction. It tells you why a rate was biased after that rate exists. It cannot stop a discriminatory rate from reaching a consumer. That is a control gap, and interpretability does not close it.

The consequences are no longer theoretical. Colorado amended Regulation 10-1-1 effective Oct. 15, 2025, extending its governance framework for external consumer data and information sources beyond life insurance to private passenger automobile insurers and naming telematics explicitly as ECDIS. Interim progress reports were due Dec. 1, 2025; compliance reports became due July 1, 2026 and recur annually. New York's Insurance Circular Letter No. 7 expects insurers to assess whether AI systems function as proxies for protected classes, and makes clear that responsibility does not transfer to a vendor. More than half of U.S. jurisdictions have adopted the NAIC Model Bulletin or substantially similar guidance.

What changed this year is examination capability rather than new obligations. The NAIC has been piloting an AI Systems Evaluation Tool across 12 states since early 2026. The pilot runs through September, with a revised version expected to be considered for adoption at the Fall National Meeting in November. It gives examiners a structured framework for reviewing insurer AI governance during market conduct and financial examinations. The question is no longer whether your program is documented. It is whether it can be demonstrated on request.

The three-layer architecture

The fix is structural rather than statistical. Put a deterministic layer between the model and the rating outcome.

Demote the model to a data generator. The predictive model calculates a risk score. It does not hold authority to issue a premium. This is a configuration decision more than a technical one, and it is the step most programs skip.

Give the rules layer the authority. Route model output into a rules engine that encodes state-specific constraints before anything reaches policy administration. Where a recommendation depends on latent proxy variables, the engine rejects it in-line. Encode the constraints per state rather than at the enterprise level, so the mapping from a control to a regulatory obligation lives in configuration instead of in prose.

Log every evaluation, not just every interception. A record written only when something is blocked cannot demonstrate coverage. A record written for every transaction turns your compliance evidence into a query rather than a project.

Two things to watch. A constraint set that is not updated as filings change becomes a source of error rather than protection. And overly restrictive constraints suppress legitimate model lift, so monitor the false-positive rate of the control layer as deliberately as the interception rate.

What the numbers showed

I tested this design and published the results in Cureus in July 2026. The trial used 10,000 synthetic multi-state personal auto underwriting profiles, generated by Monte Carlo simulation and deliberately seeded with historical proxy correlations.

Before: the standalone predictive model produced a proxy pricing disparity of 14% against protected demographic groups.

After: routing identical model outputs through the deterministic rules layer reduced that disparity to 0.14%, a relative reduction of 99%.

Three secondary findings mattered as much. The rules layer intervened on 1,976 of 10,000 transactions, or 20%. That is how much non-compliant recommendation volume a model can generate before any human sees it. Mean processing latency moved from 120 milliseconds to 125 milliseconds under simulated enterprise load, which is the finding that answers the performance objection this design reliably attracts. And a decision record was generated for 100% of transactions.

These results come from synthetic data under simulated load. They demonstrate the mechanism; they are not a production benchmark. Live environments introduce third-party API orchestration and vendor data latency that a controlled trial does not reproduce.

What to do next

Bias mitigation belongs in the architecture, not on the data science backlog. The carriers that hold up under examination will not be the ones with the most complete policy binder. They will be the ones whose systems produce the answer as a by-product of running.

Take one question into your next cross-functional review: can we produce, for any rating period, the population of AI-influenced pricing decisions and the subset where a compliance control fired? If answering it requires a project rather than a query, your evidence architecture is retrospective, and retrospective evidence is exactly what a structured examination is designed to find.

Five milliseconds is what the guardrail cost. Decide whether that is the expensive part.


Anushka Rodi

Profile picture for user Anushka Rodi

Anushka Rodi

Anushka V. Rodi is a personal lines insurance technical analyst and independent researcher. 

Her work sits at the intersection of state insurance regulation, actuarial rating logic and enterprise platform design, with a focus on governance of AI-driven risk scoring in underwriting and rating systems.

Read More