Model risk management (MRM) continues its rapid growth in the insurance sector. More insurers are adopting MRM programs and are looking to increase the efficiency and effectiveness of existing programs.
Developing and using an effective MRM system will promote better MRM performance. A basic MRM system should provide a platform for managing MRM activities, in particular tracking and managing validations. As insurers accumulate information about their models through scoring and validation processes, we believe that they can enhance their systems to gain beneficial insight across their model inventory, especially commonality of components and interactions between models.
Based on recent client work and recent industry surveys, we share below some thoughts on the key characteristics of an effective base platform, cross-inventory opportunities, and how risk managers can enhance MRM processes and systems to take advantage of these opportunities.
Basic characteristics of an MRM system
An essential starting point for insurers initiating an MRM program is to develop an inventory of their models. Because MRM programs typically encompass all of an insurer’s models (not just actuarial or risk or financial ones), the inventory can be quite large. A survey we conducted early last year indicated that more than half of respondents had more than 150 models in their inventory; a quarter had more than 450. Another survey we conducted later in the year found that MRM systems’ primary task at all insurers is to catalogue all these models.
Once catalogued, an obvious next step is to populate the system with information that helps manage the MRM process. Typically, we see the following functionality in effective systems:
1. Model documentation repository.
Model documentation is the starting point to conducting a validation. Providing access to that documentation is important for validation and continuing risk management of the model. Sometimes (typically for older models undergoing their first validation), comprehensive documentation is not available and needs to be developed. Sometimes validations point out the need for documentation to improve. Keeping track of the need to update validation either because it is inadequate or because the model has changed also should be a part of the systems’ functionality (see item 4 below).
2. Model validation document repository.
This is the most self-evident functionality. Ninety percent of respondents in our survey who had a multifunction system (i.e., systems that do more than just catalogue models) reported that it was a repository for validation or model documentation. Also of importance, as programs mature, the system needs an appropriate mechanism to update the repository with documentation from subsequent validations (presumably without losing earlier versions).
See also: Changing Business Models, ‘New’ ERM
3. Model risk scoring repository.
Though not as universal as documentation storage, storing model risk scores and the details about the model that were used to develop its score is a feature present at about two-thirds of surveyed respondent companies. Model risk scores are often used to prioritize and sequence validations, so the first score is likely developed before the model is validated. Not surprisingly, validations often shed new light on a model and can often lead to a change in score. Also, we have found some insurers have begun to revisit their earlier scores and scoring algorithms, often placing greater emphasis on models that permanently affect cash flow. The system should be capable of tracking the development of the model’s risk score because it may change over time.
4. Tracking findings needing attention and due dates for that attention.
Managing hundreds of models is likely to lead to an extensive list of findings needing attention. Keeping track of these, the party responsible for addressing them and their expected completion dates seems a natural choice for an MRM system feature. As models are being built or undergoing significant modifications, the system can be used to keep track of their progress and validation needs.
5. Emailing notification to model owners and others of coming or missed tasks.
It seems a short step from tracking as we describe above to emailing notifications and follow-ups, as required. Our survey showed that only slightly more than half of the multifunction systems have this functionality.
6. Reporting.
As with any process, reporting on MRM activity, particularly the progress of validations and issue resolution, is a necessary antecedent to managing the process. About three-quarters of the respondents have this functionality built into their system. Though only a few have developed this as a real-time reporting dashboard, the rest are working on this or planning to do so.
Cross-inventory commonalities and connections
Recognizing that MRM is still a relatively new program at many insurers, early emphasis has been on developing a system that supports initial validation efforts. However, as programs mature and systems’ basic functionality has been established, insurers should consider enhancements that could increase the overall value of their MRM program. We believe these enhancement opportunities come from better using the information in the system. In particular, they come from working across the inventory rather than one model at a time.
Different models are likely to have many assumptions in common. The system could compare assumptions across models in the inventory. If two models use different values for the same assumption, for example different values for future interest rates, it would be instructive to investigate the sources and implications of these differences. Potentially, differences are not appropriate and, if not corrected, could cause increased risk across the model inventory. A single-source model for this assumption could apply to all cases, thus reducing overall modeling costs.
Different models frequently use common parts. For example, both stress testing and ALM models may use common cash flow projection engines. Although both models should undergo their own validations, some elements of the work can be reused. In particular, with proper safeguards, multiple replication of the same calculation algorithms would be unnecessary. Often, the replication element of a validation is one of the most resource-intensive and costly aspects of the work, so avoiding duplication here could meaningfully improve efficiency.
Few if any models exist completely on their own, isolated from others in the inventory. Typically, models are fed some input from upstream models and often send some output downstream to other models. This web of connectivity can be hard to visualize, but the raw material for doing so could be available from the MRM system. Typically, systems will need some enhancement to allow insurers to mine this material, however.
Enhancing the system to enable cross-inventory gains
The next significant step in MRM’s development can come from a holistic look at the whole model inventory. Some process and system enhancements that can enable cross-inventory perspective include:
1. Model documentations standards.
Most insurers have developed a playbook or template that they expect validators to follow in conducting validations and completing validation documentation. It is not often though that we find the same attention to standards in documenting models. Standardization can benefit both the model documenters and MRM cross-inventory analysis.
2. Terminology standards.
Because many different model owners and users have developed models independent of each other over several years, it’s not surprising to encounter inconsistent terminology. Different terms often describe the same thing, and sometimes the same term describes something else. As the MRM system becomes more densely populated, a thorough review can identify inconsistencies and enable greater standardization.
3. Upstream and downstream precision.
Many validation report guidelines (and presumably good model documentation guidelines) require identification in input and output of upstream and downstream models.
It would seem a modest step to require that these identified models are cross-referenced to their place in the inventory, presumably using the same model number identification tag.
See also: Top 10 Insurtech Trends for 2017
Next steps for insurers
Insurers should bring their MRM systems up to baseline capabilities by enabling the functionalities we describe above.
As validations and model risk management activities populate the MRM system, insurers should use that information to standardize model documentation formats and develop consistent terminology. Model and validation documentation should reference upstream and downstream models using the system’s identifiers.
Insurers can then mine information contained in their MRM system to:
- Ensure consistency where required,
- Eliminate duplicative validation tasks and,
- Map their model web, eliminating unused models, improving models that need updating and carefully nurturing and managing the models that are of greatest value to the organization’s success.
