Overcoming the Challenges Posed by AI

With the competition to create the most knowledgeable AI systems, creators are getting to the point where they can’t explain how a decision was made.

Side profile of a woman against a blue backdrop with binary code lit up across her face


--Unless AI systems are trained on accurate, unbiased data by unbiased trainers, they can make faulty decisions on underwriting and claims that create legal and ethical risks.

--If great care is taken to train and review AI systems thoroughly, they can provide a host of benefits in customer service, claims and underwriting.


Artificial Intelligence (AI) is transforming the insurance industry by enabling insurers to process claims more efficiently and accurately while improving customer experience. However, the adoption of AI in claims management and underwriting, while useful, poses significant risks and challenges that insurers must address to ensure successful implementation and avoid costly mistakes. 

Big names in the tech world, such as Elon Musk and Steve Wozniak, recently penned an open letter urging labs to pause the training of AI systems for at least six months or until developers and leaders agree on better rules-of-engagement -- ensuring AI is developed to improve human life and is not weaponized or used for harm. Zurich and Lemonade are some examples of Insurers that are investigating and subsequently creating controversy around ways to leverage this technology.

Here, I will explore the risks and challenges associated with AI in claims management and underwriting, as well as strategies to mitigate them.

Risks of AI in Claims Management

AI systems are only as valuable as the data provided by their creator. Any information beyond what an AI system is fed is then inferred to create what experts refer to as hallucinations. Hallucinations can be very convincing, even though they aren’t based on good data. What constitutes “good” data from “bad” is a mix of factors such as the accuracy of the data, the perspectives, opinions and biases of the human inputting the data and who is regulating or training these individuals.

These are just a few of the risks associated with using AI. But one of the most significant is that AI can be uncontrollable. With more and more competition to create the most knowledgeable AI systems, creators are getting to the point where they can’t understand or predict their behavior or accurately explain how a decision was made. This can be problematic if humans rely on AI’s assessments to determine specific actions, such as denying or paying an insurance claim. 

Following are some other ways these risks apply to claims management:


From healthcare to recruiting, all businesses are subject to bias. The insurance industry is no exception. The U.S. Department of Commerce reports that one of the most significant risks of AI in claims management is the potential for biases, discrimination, and inaccuracy. If AI systems are trained on biased or inaccurate data, the decisions made by the AI system will be the same. This can result in unfair treatment of certain groups of individuals or incorrect claims processing, leading to reputational damage and legal liabilities.

This problem has been seen more recently regarding the use of FICO scores in the underwriting of insurance policies. Insurers are obligated to ensure that bias is removed from underwriting and claims decisions and that the information being used is accurate.

Legal/Ethical Risks

AI systems also require access to large amounts of data, including sensitive personal information, which can be vulnerable to cyber attacks and data breaches. This is a risk because it can result in significant reputational damage, legal liabilities and loss of customer trust.

There are legal and ethical risks with AI, as well. For example, an AI system denying a claim based on biased or inaccurate data could result in legal action and reputational damage. Additionally, the use of AI in claims raises ethical questions about the role of humans in decision-making and the responsibility of insurers to ensure fair treatment of their customers.

Overcoming the Challenges of AI in Claims Management

The first step in overcoming the challenges of AI in claims management and ensuring successful implementation is to review the data for accuracy and quality. It is crucial to confirm where the data came from, how it was vetted and how sensitive data is protected. This includes regular monitoring and auditing of AI systems to identify and correct errors and biases.

With the amount of content that is created online, it’s easy to pull data that stem from questionable sources. These quality checks can allow insurers to identify and correct inconsistencies in the data. Data can also change over time, so your data must be current and relevant. Routine checks will enable you to do this.

Next, you must understand how the AI system was trained and by whom. Industry experts are better-equipped to provide the knowledge AI systems are fed versus those with just tech experience.

Collaboration between humans and AI systems is another key to ensuring fair and accurate claims processing. Insurers should establish clear guidelines for when and how human intervention should occur in claims processing and confirm that humans are adequately trained to understand and collaborate with AI systems.

An independent review board inside an organization should also be implemented to test the technical efficacy of the AI system and confirm it’s working as expected. This board can also determine the moral and ethical implications and whether the system should exist in the first place.

By implementing these strategies, insurers can overcome the challenges of AI in claims management and leverage the benefits of AI to improve the efficiency and accuracy of claims processing, enhance customer experience and drive better decision-making. 

See also: 'AI' or Just 'I'? Most Adaptable Will Win!

Benefits of AI in Claims Management

While AI poses risks and challenges, it also presents significant benefits in customer service, claims and underwriting.  

Customer Service

One of the primary benefits of AI is that it can process large amounts of data quickly and accurately, reducing the time and resources required for customer service. Here is how AI can enhance the overall customer experience:

  • Use chatbots or answer bots to answer customers' questions quickly, reducing wait time
  • Submit policy changes and other simple endorsements on behalf of the customer
  • Notify the agent or customer of any outstanding items, such as payroll audits
  • Note any significant rate increases at renewal and automatically re-shop the policy
  • Create and forward policy documents, such as ID cards, declarations pages or COIs

Improved customer service can help to protect margins on commissions as a result.

Claims Management

AI can process claims end-to-end and eliminate customer or agent frustration throughout the entire claims process by:

  • Managing First Notice of Loss (FNOL) and First Report of Injury (FROI) correspondence
  • Classifying, indexing, extracting and relaying claims data into agency/carrier systems
  • Screening for potential fraud and validating eligibility
  • Calculating and setting reserves and paying claims
  • Identifying, servicing and following up on time-sensitive activity with adjusters, such as legal demands within demands packages
  • Forwarding explanations of benefits (EOBs) and policy information
  • Identifying accounts for recovery or subrogation based on the value at stake


With the challenges involved with AI and the potentially significant impact it can have on the world, there should be checks and balances to mitigate moral, ethical and legal concerns. As the AI open letter so succinctly states, rather than a race to the top, all future research by AI labs should focus on how to make the systems accurate, trustworthy and safe.

With the challenges involved with AI and the potentially significant impact it can have on the world, there should be checks and balances to mitigate moral, ethical, and legal concerns. By taking a break to establish these standards, we can be better prepared to use AI in the future and continue to revolutionize the insurance industry.

Read More