Model Validation Overview

OCC Bulletin 2000-16 outlines key model validation principles and OCC expectations for a sound model validation process. As models play an increasingly important role in banking, the need to mitigate potential model risk has also grown. This overview summarizes the guidance given in OCC Bulletin 2000-16.

A model consists of three components.

  1. Information Component: Delivers assumptions and data to the model.
  2. Processing Component: Translates inputs into estimates.
  3. Reporting Component: Translates estimates into business information.

An effective model validation process must address all three components.

General Procedures for Model Validation
There are three generic procedures applicable when validating a model: (a) independent review of logical and conceptual soundness, (b) comparison against other models, and (c) comparison of model predictions against actual results. “Regardless of a bank’s size the OCC believes that it is essential that banks develop formal policies that ensure that all of these principles are applied when circumstances warrant.” (OCC 2000-16 page 2).

Elements of Sound Validation Policy
A bank’s validation policy should include the following elements.

  1. Independent Review: Model validation should be independent of model construction.
  2. Defined Responsibility: Model validation responsibility should be formalized.
  3. Model Documentation: Procedures for use and validation should be noted.
  4. Ongoing Validation: Change control procedures should be implemented.
  5. Audit Oversight: Internal audit should ensure validation adheres to formal policy.

Validating Model Inputs
Data
Auditing data inputs should be explicitly included in the bank’s policy. Internal data should be agreed to the general ledger or other sources. External data should be checked against multiple sources. It is often extremely effective to use automated filters and the inspection of inputs by experienced personnel.
Assumptions
Assumptions derived from other models typically require that the source model also be validated. Many assumptions may be derived from general market data that is widely available. A bank may determine that specific data from its own customer base or local market offers special insight that is superior to publicly available assumptions. A clear rationale for such choices should be provided. Important assumptions should regularly be compared with actual behaviors. Results of such comparisons should be routinely provided in reports to senior management.

Validating Model Processing
Model processing includes both the mathematical and theoretical aspects of the model. The mathematical component can often be tested through comparisons with a second, well-validated “benchmark” model. The theoretical component can also be evaluated through comparisons to other models. At a minimum, validation of the theory should include a clear, non-technical description of the theory underlying the model as well as evidence that the theory has professional recognition and support.

Model Reports
The model validation process should validate the model results as well as the context of management reports produced. Model results are typically subject to “back testing” which compares the model estimates with actual results. Projections of results that are conditional upon the economic environment should also be evaluated with the actual outcomes. The context of reports should require that an executive summary be prepared for senior management, that model limitations be clearly stated, and that major assumptions should be included and highlighted. Alternative assumptions and sensitivity analysis should be provided to illustrate both a range of likely outcomes as well as the robustness of the model. The key goal is the clear communication of business information to the bank’s decision makers.

OCC Supervisory Expectations Concerning Model Validation
While model validation can be costly, using unvalidated models is potentially an unsafe and unsound practice. The OCC expects that formal policies ensure the following goals are met:

  1. Decision makers understand the meaning and limitation of a model’s results.
  2. Model results are tested against actual outcomes.
  3. The bank should audit the informational inputs to the model.
  4. The seniority of the oversight is commensurate with the materiality of the risk.
  5. Model validation must be independent from model construction.
  6. Model validation responsibilities must be clearly defined.
  7. Models must be subject to change-control procedures.

Wondering how these rules apply to you? Call us today for a free analysis and consultation at 1.800.971.1378. We’ll help you understand these complex standards and then advise you on the right approach for your financial situation — ultimately helping you maximize your financial potential.

Click here to view Model Validation Overview as a PDF.