Key takeaways

• Annex III of the AI Act treats as high-risk the AI systems used for risk assessment and pricing of natural persons in life and health insurance, with obligations applying from 2 December 2027.
• Article 86 explanations depend on a time-stamped record of every scored decision: inputs, model version, business rules, the explanation computed at scoring time and any human override.
• On tabular underwriting data, GAMs, explainable boosting machines and monotonic gradient-boosted trees usually come close to black-box accuracy, and an actuary can review and sign off each factor’s effect.
• Explanations need their own quality assurance, covering reproducibility, fidelity, stability and proxy review, with the explanation method validated alongside the model.

A policyholder whose renewal premium rises, or whose application is declined, will ask why. If the only available answer is that the model produced a score, the insurer loses that customer’s trust, however accurate the score was. Inside the business, the same unanswered question returns as a complaint and later an appeal, and eventually lands on an underwriter’s desk as a case file to rebuild by hand from old databases.

Many insurers treat explainability as a tax that regulators impose and model performance pays for. For most underwriting use cases, the evidence points the other way. On structured underwriting data, a transparent model usually costs little accuracy. The larger bill arrives in logging, documentation and validation, the infrastructure many teams have yet to build. With the AI Act’s Annex III obligations applying from 2 December 2027 following the Digital Omnibus, in-scope insurers have just over a year to put it in place. The sections below cover what regulators require, which models and explanation methods protect performance, and how to prove the explanations are true.

What the AI Act Requires from Insurers, and from Whom

Scope comes first, because it is narrower than many insurers assume. Annex III of the AI Act classifies as high-risk only the AI systems used for risk assessment and pricing of natural persons in life and health insurance. Motor, property and commercial lines fall outside that category. For those lines, EIOPA’s August 2025 opinion sets out how existing insurance legislation applies to AI.

For systems in scope, the requirements divide into documentation and runtime controls. On the documentation side, the provider maintains a risk management system under Article 9, data governance covering relevance, representativeness and bias under Article 10, and technical documentation under Article 11 and Annex IV.

The runtime obligations sit in a separate group of articles. Article 12 requires automatic logging of events across the system’s lifetime. Under Article 13, the system must be transparent enough for the deployer to interpret outputs correctly, and Article 14 adds human oversight by people who understand the system’s limits and hold the authority to override it. The insurer, as deployer, keeps logs for at least six months under Article 26 and, as a financial institution, holds them within the documentation that financial services law already requires. Before first use it also completes a fundamental rights impact assessment under Article 27. Article 86 then gives a person affected by a decision with legal or similarly significant effects the right to a clear and meaningful explanation of the role the AI system played in it.

Explanation duties already exist today. GDPR Articles 15 and 22 give people access to meaningful information about the logic involved in automated decisions, and in Dun & Bradstreet Austria (2025) the Court of Justice held that this includes an explanation of the procedure and principles applied, in a form the person can understand. Article 86 adds a right specific to high-risk AI systems on top of it.

Many insurers buy their models from vendors, and the AI Act places obligations on both the vendor, as provider, and the insurer, as deployer. The customer-facing explanation and the human oversight belong to the insurer, so vendor contracts need to guarantee access to everything those two duties depend on.

The architectural consequence follows from Article 86. A model retrained four times since last spring has lost the state it scored with back then, so the explanation has to be captured at the moment of the decision. Every scored decision therefore needs a time-stamped record containing:

• the inputs as received at scoring time,
• the model version,
• the business rules applied,
• the explanation as computed at scoring time,
• any human override.

How Solvency II Model Governance Maps to the AI Act

Much of this will look familiar to a Chief Actuary. Solvency II requires an effective system of governance under Article 41 and risk management covering underwriting and reserving under Article 44. Article 48 asks the actuarial function for an opinion on the overall underwriting policy, an opinion that becomes hard to give once the model’s logic is opaque. Where models feed an internal model for the Solvency Capital Requirement, Articles 120 to 125 add the use test, statistical quality standards, regular validation and documentation.

The overlap is substantial, and the two regimes still ask different questions. Solvency II is prudential: it asks whether the insurer understands and controls the risk its models pose to its own solvency. The AI Act protects fundamental rights and asks whether the affected individual can be told why a decision went the way it did. Both cover governance, documentation and validation. The AI Act adds per-decision logging, per-decision explanation, bias examination of training data and the impact assessment.

That shared ground gives insurers a head start. Extending the existing model inventory and validation framework to cover the AI Act additions lets one process serve both regimes, and EIOPA’s opinion frames AI governance as an interpretation of existing sectoral law.

SHAP, LIME and Interpretable Models for Underwriting at Production Scale

The next decision concerns the method that produces the stored explanation. The methods named most often behave very differently in production. For tree ensemble models, TreeSHAP computes exact Shapley values fast enough to run at scoring time and store alongside each decision. Model-agnostic KernelSHAP is far slower and relies on sampling, which is hard to justify for high-volume, per-decision use.

SHAP has a known weakness with correlated features, such as vehicle value and vehicle age, or weight and body mass index. It can split credit between overlapping variables in ways that are difficult to state honestly to a customer.

LIME has a different problem. It fits a local surrogate model on random perturbations of the input, so the same case can produce a different explanation on a different run. That inconsistency is hard to defend for a regulated decision. LIME stays useful for exploration, while the record needs a method that returns the same answer every time.

The strongest option for a regulated record is a model whose structure is the explanation. Generalised additive models and explainable boosting machines express each factor’s effect as a curve that an actuary can plot and sign off after review. Gradient-boosted trees with monotonic constraints sit between the two approaches. They are more flexible, and they can still guarantee, for example, that a premium never falls as claims history worsens.

Does Explainability Reduce Model Accuracy in Insurance Pricing?

The assumed accuracy trade-off comes largely from image and language tasks, where deep models hold a measurable accuracy lead. Underwriting data behaves differently. It is tabular, typically a few dozen to a few hundred features, most of them with established actuarial meaning. On data like this, the accuracy difference between well-built interpretable models and black boxes is often small. Cynthia Rudin made this argument influentially in 2019, and the developers of explainable boosting machines (Lou, Caruana and colleagues) report accuracy often on par with state-of-the-art black-box models. Insurance pricing also ran on generalised linear models (GLMs) for decades, and GLMs are interpretable by construction.

Where interpretable models do fall behind, the cause usually traces back to interactions between variables. Explainable boosting machines can include a small number of selected pairwise interactions and remain readable. Monotonic constraints tend to cost little accuracy and can improve out-of-sample stability, because they stop the model from learning noise that contradicts domain logic.

The performance that counts is the model’s behaviour a year after deployment, under drift, which a launch-day test-set score only partly predicts. A model that fails validation stays out of production, so its test accuracy delivers zero business value.

Teams can therefore build the interpretable model first as a benchmark and move to a more complex model only if the measured uplift justifies the extra explanation, validation and oversight burden.

One Decision Record for Supervisors, Internal Audit and Customers

Whichever model wins that comparison, three audiences will read its logged record in different ways. Supervisors and model validators need global behaviour: feature effects, stability across customer segments, performance over time, data lineage and change history.

Internal audit reads the same record for reconstructability. Given a decision ID, the team should be able to retrieve the inputs, model version, rules, stored explanation and override, and reproduce the identical score. An unreproducible score leaves the explanation unverifiable as well.

For customers, the record has to yield something else again: the few factors that mattered most, stated in plain language, along with what they could change or contest and how to request human review. A chart of contribution values means little to a policyholder. A sentence such as „your premium rose mainly because of two claims in the last three years” gives them something to act on.

Producing that sentence requires a translation layer that maps model features to plain-language reason statements approved by compliance, and testing confirms that it surfaces every factor that drove the decision. The customer explanation can be simpler than the technical one, as long as it stays faithful to it.

How to Validate That AI Explanations Are Faithful to the Model

Faithfulness is what makes the whole record worth keeping. A plausible explanation that misrepresents the model misleads every audience that reads it. A 2020 study by Slack and colleagues showed that post-hoc methods such as LIME and SHAP can be deliberately manipulated to hide a model’s reliance on sensitive features, so explanations need a quality assurance process of their own, built into the release cycle:

• Reproducibility: the same input and model version produce the same explanation, tested automatically on every release.
• Fidelity: perturbing or removing the top-ranked factors moves the score in the direction the explanation implies.
• Stability: near-identical applicants receive similar explanations.
• Proxy review: variables that correlate with protected characteristics, such as postcode, are checked for undue weight.
• Sample review: underwriters and compliance review a regular sample of customer-facing explanations, with complaints tagged to the explanation that preceded them.
• Method versioning: the independent validation function versions and validates the explanation method alongside the model, as a component of it.

Proxy review carries particular weight in the EU. Gender-based pricing has been prohibited since December 2012, following the 2011 Test-Achats ruling, and a model can rediscover it through proxies even when gender is absent from the feature set.

All of these checks feed one final test, and the December 2027 deadline gives in-scope insurers just over a year to pass it. Take a decision the system made twelve months ago and explain it to the affected person today. Most teams that run it find that the logging around the model is what’s missing, which makes logging architecture the first thing to fix.

FAQ

Which insurance AI systems are high-risk under the AI Act?

The high-risk category covers AI systems used for risk assessment and pricing of natural persons in life and health insurance, as listed in Annex III of the AI Act. Motor, property and commercial lines sit outside it, and EIOPA’s August 2025 opinion describes how existing insurance law governs AI in those lines. After the Digital Omnibus, the Annex III obligations take effect on 2 December 2027.

What does Article 86 of the AI Act mean for insurers?

Article 86 entitles a person affected by a high-risk AI decision with legal or similarly significant effects to a clear and meaningful explanation of the role the system played. For an insurer, that means storing a time-stamped record for every scored decision: inputs, model version, business rules, the explanation computed at scoring time and any human override. A retrained model can only describe its current state, so the explanation must be captured when the decision happens.

Should insurers use SHAP or LIME to explain underwriting models?

TreeSHAP is the stronger choice for production underwriting on tree ensemble models, because it computes exact Shapley values quickly enough to log with each decision. LIME builds its answer from random perturbations, so two runs on one case can disagree, which confines it to exploratory work. KernelSHAP is too slow for high-volume scoring. Teams using SHAP should also watch correlated features, where credit can be divided between variables in confusing ways.

Does explainable AI reduce accuracy in insurance pricing?

Usually only by a small margin. Underwriting data is tabular, with a few dozen to a few hundred features that mostly carry established actuarial meaning, and on such data well-built interpretable models tend to come close to black boxes. Insurance pricing relied on interpretable GLMs for decades. Any remaining difference usually comes from variable interactions, which explainable boosting machines can capture with a few selected pairwise terms while staying readable.

How does the AI Act relate to Solvency II for insurance models?

The two regimes overlap on governance, documentation and validation, and they protect different interests. Solvency II looks at whether the insurer controls the risk its models pose to its own solvency. The AI Act looks at whether an affected individual can learn why a decision was made. Extending the existing model inventory and validation framework with per-decision logging, explanations, bias checks and the impact assessment covers both.

How can insurers test whether AI explanations are accurate?

Through a dedicated quality assurance process run with every model release. Reproducibility tests confirm that the same input and model version return the same explanation. Fidelity tests remove or perturb the top-ranked factors and check that the score moves as the explanation says. Stability tests compare near-identical applicants, and proxy reviews look for variables such as postcode that may stand in for protected characteristics like gender.