Tag Archives: model risk management

How to Improve ‘Model Risk Management’

Model risk management (MRM) continues its rapid growth in the insurance sector. More insurers are adopting MRM programs and are looking to increase the efficiency and effectiveness of existing programs.

Developing and using an effective MRM system will promote better MRM performance. A basic MRM system should provide a platform for managing MRM activities, in particular tracking and managing validations. As insurers accumulate information about their models through scoring and validation processes, we believe that they can enhance their systems to gain beneficial insight across their model inventory, especially commonality of components and interactions between models.

Based on recent client work and recent industry surveys, we share below some thoughts on the key characteristics of an effective base platform, cross-inventory opportunities, and how risk managers can enhance MRM processes and systems to take advantage of these opportunities.

Basic characteristics of an MRM system

An essential starting point for insurers initiating an MRM program is to develop an inventory of their models. Because MRM programs typically encompass all of an insurer’s models (not just actuarial or risk or financial ones), the inventory can be quite large. A survey we conducted early last year indicated that more than half of respondents had more than 150 models in their inventory; a quarter had more than 450. Another survey we conducted later in the year found that MRM systems’ primary task at all insurers is to catalogue all these models.

Once catalogued, an obvious next step is to populate the system with information that helps manage the MRM process. Typically, we see the following functionality in effective systems:

1. Model documentation repository.

Model documentation is the starting point to conducting a validation. Providing access to that documentation is important for validation and continuing risk management of the model. Sometimes (typically for older models undergoing their first validation), comprehensive documentation is not available and needs to be developed. Sometimes validations point out the need for documentation to improve. Keeping track of the need to update validation either because it is inadequate or because the model has changed also should be a part of the systems’ functionality (see item 4 below).

2.  Model validation document repository.

This is the most self-evident functionality. Ninety percent of respondents in our survey who had a multifunction system (i.e., systems that do more than just catalogue models) reported that it was a repository for validation or model documentation. Also of importance, as programs mature, the system needs an appropriate mechanism to update the repository with documentation from subsequent validations (presumably without losing earlier versions).

See also: Changing Business Models, ‘New’ ERM  

3. Model risk scoring repository.

Though not as universal as documentation storage, storing model risk scores and the details about the model that were used to develop its score is a feature present at about two-thirds of surveyed respondent companies. Model risk scores are often used to prioritize and sequence validations, so the first score is likely developed before the model is validated. Not surprisingly, validations often shed new light on a model and can often lead to a change in score. Also, we have found some insurers have begun to revisit their earlier scores and scoring algorithms, often placing greater emphasis on models that permanently affect cash flow. The system should be capable of tracking the development of the model’s risk score because it may change over time.

4. Tracking findings needing attention and due dates for that attention.

Managing hundreds of models is likely to lead to an extensive list of findings needing attention. Keeping track of these, the party responsible for addressing them and their expected completion dates seems a natural choice for an MRM system feature. As models are being built or undergoing significant modifications, the system can be used to keep track of their progress and validation needs.

5. Emailing notification to model owners and others of coming or missed tasks.

It seems a short step from tracking as we describe above to emailing notifications and follow-ups, as required. Our survey showed that only slightly more than half of the multifunction systems have this functionality.

6. Reporting.

As with any process, reporting on MRM activity, particularly the progress of validations and issue resolution, is a necessary antecedent to managing the process. About three-quarters of the respondents have this functionality built into their system. Though only a few have developed this as a real-time reporting dashboard, the rest are working on this or planning to do so.

Cross-inventory commonalities and connections

Recognizing that MRM is still a relatively new program at many insurers, early emphasis has been on developing a system that supports initial validation efforts. However, as programs mature and systems’ basic functionality has been established, insurers should consider enhancements that could increase the overall value of their MRM program. We believe these enhancement opportunities come from better using the information in the system. In particular, they come from working across the inventory rather than one model at a time.

Different models are likely to have many assumptions in common. The system could compare assumptions across models in the inventory. If two models use different values for the same assumption, for example different values for future interest rates, it would be instructive to investigate the sources and implications of these differences. Potentially, differences are not appropriate and, if not corrected, could cause increased risk across the model inventory. A single-source model for this assumption could apply to all cases, thus reducing overall modeling costs.

Different models frequently use common parts. For example, both stress testing and ALM models may use common cash flow projection engines. Although both models should undergo their own validations, some elements of the work can be reused. In particular, with proper safeguards, multiple replication of the same calculation algorithms would be unnecessary. Often, the replication element of a validation is one of the most resource-intensive and costly aspects of the work, so avoiding duplication here could meaningfully improve efficiency.

Few if any models exist completely on their own, isolated from others in the inventory. Typically, models are fed some input from upstream models and often send some output downstream to other models. This web of connectivity can be hard to visualize, but the raw material for doing so could be available from the MRM system. Typically, systems will need some enhancement to allow insurers to mine this material, however.

Enhancing the system to enable cross-inventory gains

The next significant step in MRM’s development can come from a holistic look at the whole model inventory. Some process and system enhancements that can enable cross-inventory perspective include:

1. Model documentations standards.

Most insurers have developed a playbook or template that they expect validators to follow in conducting validations and completing validation documentation. It is not often though that we find the same attention to standards in documenting models. Standardization can benefit both the model documenters and MRM cross-inventory analysis.

2. Terminology standards.

Because many different model owners and users have developed models independent of each other over several years, it’s not surprising to encounter inconsistent terminology. Different terms often describe the same thing, and sometimes the same term describes something else. As the MRM system becomes more densely populated, a thorough review can identify inconsistencies and enable greater standardization.

3. Upstream and downstream precision.

Many validation report guidelines (and presumably good model documentation guidelines) require identification in input and output of upstream and downstream models.

It would seem a modest step to require that these identified models are cross-referenced to their place in the inventory, presumably using the same model number identification tag.

See also: Top 10 Insurtech Trends for 2017  

Next steps for insurers

Insurers should bring their MRM systems up to baseline capabilities by enabling the functionalities we describe above.

As validations and model risk management activities populate the MRM system, insurers should use that information to standardize model documentation formats and develop consistent terminology. Model and validation documentation should reference upstream and downstream models using the system’s identifiers.

Insurers can then mine information contained in their MRM system to:

  • Ensure consistency where required,
  • Eliminate duplicative validation tasks and,
  • Map their model web, eliminating unused models, improving models that need updating and carefully nurturing and managing the models that are of greatest value to the organization’s success.

How to Manage ‘Model Risk’ (and Win)

One of the fastest-growing concerns on insurers’ enterprise risk agenda is managing model risk. From being a phrase that primarily actuaries and other modelers used, “model risk” has become a major focus of regulators and the subject of intense activity and debate at insurers. How model risk management has evolved from ad hoc efforts to its current stage is an interesting story. But more interesting still is what we believe could be its next stage – generating measurable business value.

Generating measurable business value is model risk management’s next developmental stage.

Ad Hoc

Organizing and using experience to predict future claims is core to the business of insurance. Recognizing the importance of models, insurers and industry professionals, particularly actuaries, have long incorporated model reviews into their work.

As new models were introduced or changes made to existing ones – especially if third-party systems were involved – insurers were careful to ensure consistency between old and new models. Additionally, internal and external auditors’ procedures recognized the risk that models entail and incorporated verification and testing in their processes.

See Also: Secret Sauce for New Business Models?

What distinguishes this earliest stage is not that model risk was ignored but rather that model risk management was dispersed and generally informal. Practices differed across the industry, across different types of professional organizations and across different parts and functions within an insurer. Standards for documentation, both of the models and the validation process, were largely absent. Typically, not all models were reviewed. Establishing a comprehensive inventory of all significant models was not the norm. Likewise, it was not common for insurers to follow consistent procedures to validate models across the enterprise.

Reactive

Although a comprehensive guide to help banks mitigate potential risks arising from reliance on models was available as early as 2000, concerted attention to the issue in insurance can be dated to the Great Recession and its aftermath. In reaction to the events of 2008/2009, regulators and insurers themselves revisited their risk management processes and governance.

The U.S. Federal Reserve Board took the lead in promulgating new requirements for the banking sector, including supervisory guidance on model risk management issued in 2011. Many insurers, especially those designated as systematically important financial institutions (SIFIs), have been working to adopt these guidelines. In 2012, the North American CRO Council released its model validation principles for risk and capital models, which included eight core validation principles. For insurers operating in Europe, Solvency II provided the potential to use an internal model to establish their capital requirements. To take advantage of this opportunity, insurers needed to adhere to model validation expectations prescribed by regulators. In the U.S., the ORSA Guidance Manual requires insurers to describe their validation process.

Reacting to the 2008/2009 crisis and regulators’ demands, insurers began to establish the key elements of an enterprisewide model risk management program:

  • Governance and independence policies;
  • An inventory and risk assessment of all significant models; and
  • Documentation and validation standards.

Only after these basic building blocks had been put in place did insurers developed the practical experience to begin their transition to the next, active stage.

Active

The reactive stage and the beginning of the active stage effectively started in 2014. In the early months of that year, PwC conducted a survey of 36 insurers operating in the U.S. The survey provided the opportunity for participants to assess their programs across 10 dimensions characterizing the key elements of a monitoring and reporting mechanism (MRM) process. Modal responses across these dimensions were typically “weak” or “developing.” Almost all insurers admitted they had work to do and indicated that they had plans in place to improve their processes.

In the intervening two years, we have observed a significant investment in MRM capabilities. In the absence of detailed insurance-focused regulatory guidelines, most insurers have shaped their developments to best fit their own circumstances. For example, while there has been a near-uniform increase in resources allocated to MRM, how insurers deploy these resources has differed significantly. Some have formed large centralized model management functions, and others have allocated most of the validation responsibility to business units. How the responsibilities are dispersed across risk, actuarial, compliance and audit functions vary considerably. We expect that most of these differences are attempts to fit the task to the insurer’s existing structure and culture.

Likewise, we have seen insurers, both individually and as a group, more actively develop procedures that better fit the unique circumstances of the insurance sector instead of banking or financial services in general. Three areas in which the insurance sector is increasing its attention are:

  1. Incorporating the unique aspects of actuarial models and the development of standards by actuarial professional organizations;
  2. Emphasizing the process of assumption setting and the governance of this process; and
  3. Emphasizing monitoring and benchmarking necessitated by the long time frame and the lack of market data to measure the performance of many insurance models.

Productive

Recent discussions with forward-thinking insurance company executives and board members leads us to think a fourth stage may be next. The common theme is recognition that an insurer’s key asset is the information it possesses and the models it has developed to turn this information into support for profit-generating decisions. Seen in this light, models are not inconveniences substituting for “real” data. Rather, they are the machinery that insurers use to turn their raw materials (data) into salable, profitable costumer solutions.

See Also: How to Remove Fear in Risk Management

Model risk management then becomes the mechanism to ensure this machinery is performing at its best. This includes the normal activities that one would associate with maintenance, like finding and correcting inadequate performance. But, it also provides a way to determine how better machinery can be developed and brought online.

In many respects, the transition to this stage mirrors the transition that has occurred in risk management in general. Not too long ago, risk management was seen as a strictly defensive activity. It was more about saying “no” than finding the right opportunities to say “yes.” Now, risk management is seen as an important strategic activity that plays a central role in an insurer’s deployment of capital and its selection of growth opportunities.

Putting models and the data that feeds them at the center of an insurer’s value creation engine, instead of at its periphery, provides a new perspective. And, by moving model risk management to the productive stage, insurers can better use this new perspective to address customer expectations in an information-rich environment.

Implications

  • Model risk management is no longer an ad hoc or reactive activity. An active approach is now a necessity to meet internal and external stakeholder demands.
  • Insurers are attempting to develop model risk management practices that fit the needs of their industry. They will need to continually communicate to regulators, standards setters and other stakeholders how the business of insurance has unique characteristics compared with elsewhere in financial services.
  • Models are among insurers’ greatest assets, and the machinery that they use to turn data into salable, profitable costumer solutions. Putting models and the data that feeds them at the center of value creation can provide new perspectives that better address customer expectations. Model risk management becomes the tool to keep this machinery productive.

Modernization: CRO Faces New ‘Unknowns’

Internal and external demands have resulted in the clarification and expansion of the role of the chief risk officer and the risk management function. Internally, senior management and the board see the merit of using key risk information. Ensuring the company is managed within its risk appetite enables it to best utilize its resources to take advantage of changing competitive needs and strategic opportunities. Externally, U.S. and global regulators are articulating clear expectations for the role of the CRO and governance of the risk function, as well as the role of the board in risk management and the CRO’s and risk function’s relationship with the board. These demands emphasize the need for clear policies and processes with appropriate documentation and governance.

As little as 10 years ago, the risk function was novel at most companies, and there were almost as many models of how to organize and manage the function as there were insurers. This has changed. Leading practice is becoming clearer, and expectations are now more consistent and defined. However, boards and regulators are increasingly inquiring about new “unknowns”: data security, cyber terrorism, reputational risk and competitive obsolescence. All of these also fall under the CRO’s purview and increase demands on risk resources.

The case for change

The risk function is the newest among the direct stakeholders that insurance modernization directly affects, and there are a number of important implications and outcomes.

  • No existing “pipes” – For the majority of North American risk functions, many risk calculations and resulting reports are very recent creations. Very few have a solid network of pipes that transmit data and input through models and calculations onward to result in verifiable and controlled information. Therefore, compared with many other functions that modernization affects, the risk function does not need to dismantle existing pipes. However, it is critically important that, as insurers plan and develop these new pipes, they do so in cooperation with other stakeholders. If they do not, then the risk function may find itself unnecessarily tearing up what should be a common roadway.
  • From build to oversee – While internal and external changes affect all stakeholders, the risk function is unique in that its very nature also is changing. When the risk function originally came into being, it was the CRO’s and his staff’s responsibility to create the models and capability needed to support the function. Now, as risk infrastructure takes shape, management, boards and other stakeholders are asking the CRO and risk function to play a key role in governance and control. This brings into question how best to manage and oversee both the risk and overall corporate infrastructure. Can and should these be responsibilities of the risk function, and, if not, who should be responsible for managing this infrastructure?
  • Process and documentation – Much of the newly built infrastructure was constructed quickly and in a “learn by doing” mode. Much of it is parallel to but not coordinated with activity in other areas, especially actuarial. As companies have mapped processes and documented assumptions, models and output, functional overlaps have become clearer. In many cases, clarification and resolution of the overlaps will be necessary to enable rational enterprise level mapping and non-duplicative documentation.
  • Demonstrated engagement – The CRO and risk management staff (with input from actuarial, investment, finance and others) support the foundation on which risk information is built Increasingly, the board and regulators are asking for holistic engagement in agreeing on assumptions and methodologies, not just siloed input from subject-matter experts. The risk function increasingly is being asked: Are the business managers – the first line of defense –in agreement? And, is their collective engagement substantive and verifiable?
  • Governance – As the board’s role in risk management and risk taking becomes clearer, many boards and regulators recognize the need to include major risk and strategic initiatives under the oversight umbrella. They look to the CRO to be the conduit of information between them and the insurer. This strongly suggests that the CRO should have insight into modernization initiatives that go beyond just the risk function.

In a modernized company, a synergy of efficient processes with clearly defined stakeholder expectations exists among risk, actuarial, finance and technology (RAFT). The modernized risk function will share a common foundation of data, methods and assumptions and tools and technology with the other RAFT functions. (Naturally, the risk function will have certain unique processes that build on this foundation.) Finally, enterprise compatible business management, HR, reporting and governance all channel the process to its apex: intelligent decision making.

  • Data – The organization, with significant risk input, clearly defines its data strategy via integrated information from commonly recognized sources. The goal of this strategy is information that users can extract and manipulate with minimal manual intervention at a sufficient level of detail to allow for on-demand analysis.
  • Methods and analysis – Modern risk organizations emphasize robust methods and analysis, particularly the utilization of different approaches to arrive at insight from more than one perspective. Key to proper utilization of multiple methods is confidence that different outcomes are not the result of inconsistent inputs but rather truly reflect new insight.
  • Tools and technology – Up-to-date tools and technology help the risk function gather, analyze and share information faster, more accurately and more transparently than ad hoc end-user computing analysis. With modern tools and technology, risk personnel can devote the majority of their time to understanding and managing risk rather than programming and running risk models.
  • Stress testing – Stress testing has become a key weapon in the risk management arsenal. Test results convey risk information to senior management, the board and regulators. Resulting impacts on capital under stress scenarios become key to capital planning and calibrating economic capital (EC) models. Moreover, these tests are fully integrated in financial planning and the finance function’s agenda.
  • EC/Capital modeling – Economic capital calculations continue to be an important tool for decisions at all levels, from strategic to micro-level asset trading and product design. A modernized organization fully integrates these models with key actuarial activities, and the process and results help the company more effectively plan for and manage risk. Results are available quickly, and efficiency of the process allows for extensive “what if” testing.
  • Validation – A comprehensive model risk management structure is in place. The company routinely validates new models and model changes. Assumption consistency is transparent across risk, actuarial and finance. The company verifies data integrity and uses a model inventory to weed out duplication and overlap. Savings more than pay for model risk management (MRM) costs.
  • Human capital – Risk functions employ more inquisitive and analytical analysts. The emphasis is on managing risk, not running models. A significant portion of the group devotes its time to understanding emerging trends and investigating potential new threats to the organization. Clear organizational design facilitates working in a collaborative manner with other control functions and business managers.
  • Governance – Risk plays a key role in governance and risk appetite is well established. Decision making throughout the organization incorporates risk in a transparent manner. This is in large part because of confidence in risk output because data and input is consistent with finance and actuarial analytics, models are validated and senior management and the board understand key assumptions and limitations.

The benefits

Realizing ERM’s promise requires more than just complex economic capital and value at risk (VAR) models. It requires confidence in these models and an understanding of their key assumptions and limitations. This confidence and understanding need to be pervasive – from risk, finance and actuarial personnel themselves, through line of business leadership, up to senior management and the board.

With a modernized platform in place, CROs and risk functions can turn their attention to managing risk, not calculating and reconciling numbers, as well as providing management and board with the best tools for intelligent decision making, confidence in capital deployment and competitive strategies consistent with risk appetite and capacity.

Critical success factors

Plan ahead and in concert with other stakeholders. The risk function is in the unique position of not having to dismantle infrastructure, but it definitely does need to build on it. The function’s relative youth and lack of legacy encumbrances mean it is in an ideal position to be a leader in modernization initiatives.

Moreover, the risk function has both an opportunity and an obligation to raise concerns about the risks involved in modernizing in an uncoordinated way or the risk to the insurer’s competitiveness from not modernizing at all.

Call to action – Next steps

Look for quick wins, like faster processing, more transparency, deeper insight, but stay true to the long-term plan. Some of these quick wins can be cost savings opportunities. For example, an inventory of documented models can reduce the number of models (and associated maintenance cost) by weeding out redundancies. In addition, the company can streamline internal reports when all areas use the same foundational data and calculations. Moreover, the company may be able to rationalize multi-jurisdictional, external and regulatory reporting.

The Case for Modernizing Insurance

Several drivers of change are compelling insurance companies to re-evaluate and modernize all aspects of their business model and operations. These drivers include new and rigorous expectations from regulators and standards, increasing demands for more relevant and useful information, improvements in analytics and the need for operational transformation.

The modernization creates considerable expectations for finance, risk and actuarial functions, and potentially significant impacts to business strategy, investor education, internal controls, valuation models and the processes and systems underlying each – as well as other fundamental aspects of the insurance business. Accordingly, insurers need more sophisticated financial reporting, risk management and actuarial analysis to address complex measurement and disclosure changes, regulatory requirements and market expectations.

Three key areas to look at:

Regulation and reporting

Changes in regulatory and reporting requirements will place greater demands on finance, risk and actuarial functions. Issues include:

  • Changing global and federal regulation (e.g., Federal Insurance Office, Federal Reserve oversight)
  • ComFrame, a common framework for international supervision.
  • Principle-based reserving
  • Own Risk and Solvency Assessment (ORSA), the Solvency II initiative that defines a set of processes for decision-making and strategic analysis
  • Solvency reporting measures
  • Insurance contract accounting

Information and analytics

Stakeholders are demanding more information, and boards and the C-suite need new and more relevant metrics to manage their businesses. Issues include:

  • Economic capital
  • Embedded value
  • Customer analysis and behavioral simulation
  • New product and changing underwriting parameters

Operational transformation

Those in charge of governance are demanding that the data they use to manage risk and make decisions be more reliable and economical. Issues include:

  • Updated target operating models
  • Centers of excellence
  • Enterprise risk management (ERM), model risk management and governance
  • New framework from the Committee of Sponsoring Organizations (COSO), a joint initiative of five private-sector organizations that provides thought leadership on ERM, internal controls and fraud deterrence
  • Optimization of controls, and efficiency studies

These drivers of change, which affect every facet of the business — from processes, systems and controls to employees and investor relations — have significant overlaps, and insurers cannot deal with them in isolation. To meet emerging challenges and requirements, simply adding processes or making one-off, isolated changes will not work.

Systems, data and modeling will have to improve, and the finance, actuarial and risk functions will need to work together more closely and effectively than they ever have before to meet new demands both individually and as a whole.

Moreover, all of this change is imminent: Over the next five years, leading companies will separate themselves from their competitors by fully developing and implementing consistent data, process, technology and human resource strategies that enable them to meet these new requirements and better adapt to changing market conditions.

The insurers that wind up ahead of the game will excel at creating timely, relevant and reliable management information that will provide them a strategic advantage. Legacy processes and systems will not be sufficient to address pending regulatory and reporting changes or respond to market opportunities, competitive threats, economic pressures and stakeholder expectations. Companies that do not respond effectively will struggle with sub-par operating models, higher capital costs, compliance challenges and an overall lack of competitiveness.

In subsequent articles, we will take a closer look at those leaders/business units that need to modernize.

 Eric Trowbridge, a senior manager, contributed to this article.