Tag Archives: mrm

How to Improve ‘Model Risk Management’

Model risk management (MRM) continues its rapid growth in the insurance sector. More insurers are adopting MRM programs and are looking to increase the efficiency and effectiveness of existing programs.

Developing and using an effective MRM system will promote better MRM performance. A basic MRM system should provide a platform for managing MRM activities, in particular tracking and managing validations. As insurers accumulate information about their models through scoring and validation processes, we believe that they can enhance their systems to gain beneficial insight across their model inventory, especially commonality of components and interactions between models.

Based on recent client work and recent industry surveys, we share below some thoughts on the key characteristics of an effective base platform, cross-inventory opportunities, and how risk managers can enhance MRM processes and systems to take advantage of these opportunities.

Basic characteristics of an MRM system

An essential starting point for insurers initiating an MRM program is to develop an inventory of their models. Because MRM programs typically encompass all of an insurer’s models (not just actuarial or risk or financial ones), the inventory can be quite large. A survey we conducted early last year indicated that more than half of respondents had more than 150 models in their inventory; a quarter had more than 450. Another survey we conducted later in the year found that MRM systems’ primary task at all insurers is to catalogue all these models.

Once catalogued, an obvious next step is to populate the system with information that helps manage the MRM process. Typically, we see the following functionality in effective systems:

1. Model documentation repository.

Model documentation is the starting point to conducting a validation. Providing access to that documentation is important for validation and continuing risk management of the model. Sometimes (typically for older models undergoing their first validation), comprehensive documentation is not available and needs to be developed. Sometimes validations point out the need for documentation to improve. Keeping track of the need to update validation either because it is inadequate or because the model has changed also should be a part of the systems’ functionality (see item 4 below).

2.  Model validation document repository.

This is the most self-evident functionality. Ninety percent of respondents in our survey who had a multifunction system (i.e., systems that do more than just catalogue models) reported that it was a repository for validation or model documentation. Also of importance, as programs mature, the system needs an appropriate mechanism to update the repository with documentation from subsequent validations (presumably without losing earlier versions).

See also: Changing Business Models, ‘New’ ERM  

3. Model risk scoring repository.

Though not as universal as documentation storage, storing model risk scores and the details about the model that were used to develop its score is a feature present at about two-thirds of surveyed respondent companies. Model risk scores are often used to prioritize and sequence validations, so the first score is likely developed before the model is validated. Not surprisingly, validations often shed new light on a model and can often lead to a change in score. Also, we have found some insurers have begun to revisit their earlier scores and scoring algorithms, often placing greater emphasis on models that permanently affect cash flow. The system should be capable of tracking the development of the model’s risk score because it may change over time.

4. Tracking findings needing attention and due dates for that attention.

Managing hundreds of models is likely to lead to an extensive list of findings needing attention. Keeping track of these, the party responsible for addressing them and their expected completion dates seems a natural choice for an MRM system feature. As models are being built or undergoing significant modifications, the system can be used to keep track of their progress and validation needs.

5. Emailing notification to model owners and others of coming or missed tasks.

It seems a short step from tracking as we describe above to emailing notifications and follow-ups, as required. Our survey showed that only slightly more than half of the multifunction systems have this functionality.

6. Reporting.

As with any process, reporting on MRM activity, particularly the progress of validations and issue resolution, is a necessary antecedent to managing the process. About three-quarters of the respondents have this functionality built into their system. Though only a few have developed this as a real-time reporting dashboard, the rest are working on this or planning to do so.

Cross-inventory commonalities and connections

Recognizing that MRM is still a relatively new program at many insurers, early emphasis has been on developing a system that supports initial validation efforts. However, as programs mature and systems’ basic functionality has been established, insurers should consider enhancements that could increase the overall value of their MRM program. We believe these enhancement opportunities come from better using the information in the system. In particular, they come from working across the inventory rather than one model at a time.

Different models are likely to have many assumptions in common. The system could compare assumptions across models in the inventory. If two models use different values for the same assumption, for example different values for future interest rates, it would be instructive to investigate the sources and implications of these differences. Potentially, differences are not appropriate and, if not corrected, could cause increased risk across the model inventory. A single-source model for this assumption could apply to all cases, thus reducing overall modeling costs.

Different models frequently use common parts. For example, both stress testing and ALM models may use common cash flow projection engines. Although both models should undergo their own validations, some elements of the work can be reused. In particular, with proper safeguards, multiple replication of the same calculation algorithms would be unnecessary. Often, the replication element of a validation is one of the most resource-intensive and costly aspects of the work, so avoiding duplication here could meaningfully improve efficiency.

Few if any models exist completely on their own, isolated from others in the inventory. Typically, models are fed some input from upstream models and often send some output downstream to other models. This web of connectivity can be hard to visualize, but the raw material for doing so could be available from the MRM system. Typically, systems will need some enhancement to allow insurers to mine this material, however.

Enhancing the system to enable cross-inventory gains

The next significant step in MRM’s development can come from a holistic look at the whole model inventory. Some process and system enhancements that can enable cross-inventory perspective include:

1. Model documentations standards.

Most insurers have developed a playbook or template that they expect validators to follow in conducting validations and completing validation documentation. It is not often though that we find the same attention to standards in documenting models. Standardization can benefit both the model documenters and MRM cross-inventory analysis.

2. Terminology standards.

Because many different model owners and users have developed models independent of each other over several years, it’s not surprising to encounter inconsistent terminology. Different terms often describe the same thing, and sometimes the same term describes something else. As the MRM system becomes more densely populated, a thorough review can identify inconsistencies and enable greater standardization.

3. Upstream and downstream precision.

Many validation report guidelines (and presumably good model documentation guidelines) require identification in input and output of upstream and downstream models.

It would seem a modest step to require that these identified models are cross-referenced to their place in the inventory, presumably using the same model number identification tag.

See also: Top 10 Insurtech Trends for 2017  

Next steps for insurers

Insurers should bring their MRM systems up to baseline capabilities by enabling the functionalities we describe above.

As validations and model risk management activities populate the MRM system, insurers should use that information to standardize model documentation formats and develop consistent terminology. Model and validation documentation should reference upstream and downstream models using the system’s identifiers.

Insurers can then mine information contained in their MRM system to:

  • Ensure consistency where required,
  • Eliminate duplicative validation tasks and,
  • Map their model web, eliminating unused models, improving models that need updating and carefully nurturing and managing the models that are of greatest value to the organization’s success.

Modernization: CRO Faces New ‘Unknowns’

Internal and external demands have resulted in the clarification and expansion of the role of the chief risk officer and the risk management function. Internally, senior management and the board see the merit of using key risk information. Ensuring the company is managed within its risk appetite enables it to best utilize its resources to take advantage of changing competitive needs and strategic opportunities. Externally, U.S. and global regulators are articulating clear expectations for the role of the CRO and governance of the risk function, as well as the role of the board in risk management and the CRO’s and risk function’s relationship with the board. These demands emphasize the need for clear policies and processes with appropriate documentation and governance.

As little as 10 years ago, the risk function was novel at most companies, and there were almost as many models of how to organize and manage the function as there were insurers. This has changed. Leading practice is becoming clearer, and expectations are now more consistent and defined. However, boards and regulators are increasingly inquiring about new “unknowns”: data security, cyber terrorism, reputational risk and competitive obsolescence. All of these also fall under the CRO’s purview and increase demands on risk resources.

The case for change

The risk function is the newest among the direct stakeholders that insurance modernization directly affects, and there are a number of important implications and outcomes.

  • No existing “pipes” – For the majority of North American risk functions, many risk calculations and resulting reports are very recent creations. Very few have a solid network of pipes that transmit data and input through models and calculations onward to result in verifiable and controlled information. Therefore, compared with many other functions that modernization affects, the risk function does not need to dismantle existing pipes. However, it is critically important that, as insurers plan and develop these new pipes, they do so in cooperation with other stakeholders. If they do not, then the risk function may find itself unnecessarily tearing up what should be a common roadway.
  • From build to oversee – While internal and external changes affect all stakeholders, the risk function is unique in that its very nature also is changing. When the risk function originally came into being, it was the CRO’s and his staff’s responsibility to create the models and capability needed to support the function. Now, as risk infrastructure takes shape, management, boards and other stakeholders are asking the CRO and risk function to play a key role in governance and control. This brings into question how best to manage and oversee both the risk and overall corporate infrastructure. Can and should these be responsibilities of the risk function, and, if not, who should be responsible for managing this infrastructure?
  • Process and documentation – Much of the newly built infrastructure was constructed quickly and in a “learn by doing” mode. Much of it is parallel to but not coordinated with activity in other areas, especially actuarial. As companies have mapped processes and documented assumptions, models and output, functional overlaps have become clearer. In many cases, clarification and resolution of the overlaps will be necessary to enable rational enterprise level mapping and non-duplicative documentation.
  • Demonstrated engagement – The CRO and risk management staff (with input from actuarial, investment, finance and others) support the foundation on which risk information is built Increasingly, the board and regulators are asking for holistic engagement in agreeing on assumptions and methodologies, not just siloed input from subject-matter experts. The risk function increasingly is being asked: Are the business managers – the first line of defense –in agreement? And, is their collective engagement substantive and verifiable?
  • Governance – As the board’s role in risk management and risk taking becomes clearer, many boards and regulators recognize the need to include major risk and strategic initiatives under the oversight umbrella. They look to the CRO to be the conduit of information between them and the insurer. This strongly suggests that the CRO should have insight into modernization initiatives that go beyond just the risk function.

In a modernized company, a synergy of efficient processes with clearly defined stakeholder expectations exists among risk, actuarial, finance and technology (RAFT). The modernized risk function will share a common foundation of data, methods and assumptions and tools and technology with the other RAFT functions. (Naturally, the risk function will have certain unique processes that build on this foundation.) Finally, enterprise compatible business management, HR, reporting and governance all channel the process to its apex: intelligent decision making.

  • Data – The organization, with significant risk input, clearly defines its data strategy via integrated information from commonly recognized sources. The goal of this strategy is information that users can extract and manipulate with minimal manual intervention at a sufficient level of detail to allow for on-demand analysis.
  • Methods and analysis – Modern risk organizations emphasize robust methods and analysis, particularly the utilization of different approaches to arrive at insight from more than one perspective. Key to proper utilization of multiple methods is confidence that different outcomes are not the result of inconsistent inputs but rather truly reflect new insight.
  • Tools and technology – Up-to-date tools and technology help the risk function gather, analyze and share information faster, more accurately and more transparently than ad hoc end-user computing analysis. With modern tools and technology, risk personnel can devote the majority of their time to understanding and managing risk rather than programming and running risk models.
  • Stress testing – Stress testing has become a key weapon in the risk management arsenal. Test results convey risk information to senior management, the board and regulators. Resulting impacts on capital under stress scenarios become key to capital planning and calibrating economic capital (EC) models. Moreover, these tests are fully integrated in financial planning and the finance function’s agenda.
  • EC/Capital modeling – Economic capital calculations continue to be an important tool for decisions at all levels, from strategic to micro-level asset trading and product design. A modernized organization fully integrates these models with key actuarial activities, and the process and results help the company more effectively plan for and manage risk. Results are available quickly, and efficiency of the process allows for extensive “what if” testing.
  • Validation – A comprehensive model risk management structure is in place. The company routinely validates new models and model changes. Assumption consistency is transparent across risk, actuarial and finance. The company verifies data integrity and uses a model inventory to weed out duplication and overlap. Savings more than pay for model risk management (MRM) costs.
  • Human capital – Risk functions employ more inquisitive and analytical analysts. The emphasis is on managing risk, not running models. A significant portion of the group devotes its time to understanding emerging trends and investigating potential new threats to the organization. Clear organizational design facilitates working in a collaborative manner with other control functions and business managers.
  • Governance – Risk plays a key role in governance and risk appetite is well established. Decision making throughout the organization incorporates risk in a transparent manner. This is in large part because of confidence in risk output because data and input is consistent with finance and actuarial analytics, models are validated and senior management and the board understand key assumptions and limitations.

The benefits

Realizing ERM’s promise requires more than just complex economic capital and value at risk (VAR) models. It requires confidence in these models and an understanding of their key assumptions and limitations. This confidence and understanding need to be pervasive – from risk, finance and actuarial personnel themselves, through line of business leadership, up to senior management and the board.

With a modernized platform in place, CROs and risk functions can turn their attention to managing risk, not calculating and reconciling numbers, as well as providing management and board with the best tools for intelligent decision making, confidence in capital deployment and competitive strategies consistent with risk appetite and capacity.

Critical success factors

Plan ahead and in concert with other stakeholders. The risk function is in the unique position of not having to dismantle infrastructure, but it definitely does need to build on it. The function’s relative youth and lack of legacy encumbrances mean it is in an ideal position to be a leader in modernization initiatives.

Moreover, the risk function has both an opportunity and an obligation to raise concerns about the risks involved in modernizing in an uncoordinated way or the risk to the insurer’s competitiveness from not modernizing at all.

Call to action – Next steps

Look for quick wins, like faster processing, more transparency, deeper insight, but stay true to the long-term plan. Some of these quick wins can be cost savings opportunities. For example, an inventory of documented models can reduce the number of models (and associated maintenance cost) by weeding out redundancies. In addition, the company can streamline internal reports when all areas use the same foundational data and calculations. Moreover, the company may be able to rationalize multi-jurisdictional, external and regulatory reporting.