Each autumn, we publish our Forecast evaluation report (FER), a detailed examination of the performance of past economic and fiscal forecasts relative to the latest outturn data. In a similar vein, the Bank of England recently published a review of its forecast errors since 2010,a while the OECD has published an assessment of its forecasts since the global financial crisis.b
Like most other forecasters, the OECD found that average GDP growth came in below its country-level forecasts between 2007 and 2012, during both the global recession and recovery. It made larger forecast errors over the whole period for countries that were more exposed to global developments, with important factors including openness to international trade and prevalence of foreign-owned banks. This suggested that it had underestimated the impact of global shocks on individual countries. There were also larger errors for those countries with tighter labour and product market regulation, suggesting that it did not fully take into account the role of flexibility in the face of shocks. Its largest errors were for countries where banks entered the recession with low capital ratios, suggesting that it may also have underestimated the negative impact on an economy of banks repairing their balance sheets.
In contrast to the IMF,c the OECD concluded that larger-than-expected impacts from fiscal consolidations did not contribute much to weaker-than-expected growth. The lack of narrowing of euro area sovereign bond spreads, as the euro area crisis failed to ease as expected, had a larger impact.
The Bank of England, like us, over-forecast the recovery in real GDP growth following the financial crisis. We compared our real GDP forecast errors with the Bank’s in Box 2.1 of our December EFO. The Bank have now also published an analysis of the inflation forecast errors made in the August 2010 Inflation Report. Again like us, the Bank under-forecast inflation from mid-2010 (Chart A). The Bank’s average forecast error on inflation between the third quarter of 2010 and third quarter of 2013 was 1.4 percentage points. The average error from our June 2010 forecast was 1.2 percentage points over the same period.
The Bank estimate that energy prices, non-energy import prices and university tuition fees explain most of the errors in its inflation forecasts. The drivers of these errors were larger-than-expected increases in global oil, gas and agricultural commodity prices, higher energy distribution costs, greater impact from sterling depreciation and the large rise in tuition fees in 2012. In the 2013 FER, we identified similar factors, notably higher global commodity prices. We also believe that higher-than-expected unit labour costs contributed to our errors, as nominal rigidities in wage-setting may have meant that some of the weakness in real incomes arising from lower-than-expected productivity was felt via higher prices, as firms passed on those higher costs of production to consumers.
Chart A: CPI forecasts and outturns
Forecasters use the analysis of past forecast errors to inform future forecasts, including forecast methodology changes. The process of producing the FER allows us to learn lessons that we can apply to future forecasts. The OECD state that their review of forecast errors has led them to increase the top-down element of their country forecasts, where the central world view has more of an influence, and they are now making more use of anecdotal evidence from business contacts. Another lesson most forecasters have taken is the need to focus more on financial market developments. The OECD now makes use of financial condition indices and are working on incorporating the banking sector more fully into models, as failure to take these factors into account were found to be a major source of its forecast errors. We look closely at developments in credit conditions and bank funding markets when producing our forecasts.