No community bank has completed its regulatory requirements without an asset liability report. And no asset liability report is complete without a backtest. So, just what makes up a good backtest?
To get at the answer look back at OCC 2000 - 16, the initial regulatory standard on model validation and verification. Interestingly, you will note that the standard applies to all models used within a bank, not just to asset liability models. However, it's a good bet that community banks mainly run into the backtest in the context of an asset liability examination.
Basically, the question the backtest is trying to answer boils down to "Does your model do a good job of projecting net interest income?" So, how do we determine this? The answer is surprisingly simple.
Take the projected net interest income from your asset liability model. Adjusted for those things which were unknown at the time the model was run. Then, compare the adjusted projection with the actual results. A good fit from your asset liability model will result in an adjusted projection that closely tracks the actual net interest income earned by the bank.
Here are the main areas of adjustment:
- Growth of the Bank. It cannot be determined with certainty at the time the asset liability model is being run how much the bank might grow or shrink during that quarter.
- Product Mix Changes. Changes in the mix of bank assets or liabilities cannot be determined in advance.
- Interest Rate Changes. While the asset liability model exists to project how net interest income might shift given certain predefined shocks in interest rates, the actual interest rate environment experienced during the quarter is unknown at the time the model is run.
So, now that you've completed your backtest, or have looked at the backtest attached to your quarterly asset liability report, what's the standard to determine a good fit? The answer here is undefined. However, a good rule of thumb should be that total interest income and total interest expense, as well as net interest income, should not vary from the actuals by more than 10%. If your projection varies more than 10%, it typically suggests that your bank has gone through a period of extremely fast growth, such as with a de novo bank, your product mix has changed remarkably (not likely), or interest rates have moved sharply during the period. Often, you will see adjusted projections in your backtest that differ less than 5% from your actual net interest income.
So, after having gone through and examined changes in your bank's size, mix, and the actual change in interest rates, what if the unexplained variance still exceeds your tolerances? In this case, it's a safe bet that some of your underlying model assumptions need to be revised. While it's a good idea to periodically review your assumptions, it's a real requirement when your backtest shows large unexplained variances.
Photo provided by YtseJam Photography