Accuracy assessment of the FPB short-term forecasts - An update [ Working Paper 12-17 - 29/09/2017]

  PDF & Download

• Publication (en - 776 Kb)
• Publication (fr - 792 Kb)
• Publicatie (nl - 807 Kb)
• Slideshow (nl - 530 Kb)
• Avis du Comité scientifique - Advies van het Wetenschappelijk Comité (mix - 14 Kb)

  Authors

Ludovic Dobbelaere (A), Igor Lebrun (A)
 
A : Author, C : Contributor

  Publication type

Working Papers

The Working Paper presents a study or analysis conducted by the Federal Planning Bureau on its own initiative.

The Federal Planning Bureau is responsible, within the National Accounts Institute, for producing the macroeconomic forecasts that are used to set up the federal government budget. According to its guiding principles, the FPB is as transparent as possible about its forecasts, which not only implies that a clear description of the hypotheses is available, but also of the tools and methods used to produce the forecasts. To further enhance transparency, the FPB has – at its own initiative – regularly produced ex post assessments of the quality of its forecasts in the past. Since the Law of 21 December 1994 creating the National Accounts Institute was amended by the Law of 28 February 2014, this evaluation has become mandatory and must be submitted to a scientific committee. The result of that evaluation shall also be made public and taken into account appropriately in upcoming macroeconomic forecasts.

Within this new legal framework, the present working paper provides an update of the post-mortem assessment of the annual forecasts from 2012. Compared to this previous study, the sample has been extended by six additional years and the number of evaluated variables has been increased. Moreover, this paper examines to what extent the observed forecast errors on economic growth and inflation are due to errors made on the exogenous assumptions related to the international environment.

Although several hundreds of variables are forecast in an economic budget, only the most relevant ones to set up the government budget were selected for the evaluation. Real growth in GDP and expenditure components were chosen as the most important indicators of economic activity. Analogously, growth in the consumer price index, in the GDP deflator and in the deflators of the expenditure components are the price variables that will be covered in this analysis. Growth in nominal GDP and its main income components were selected because they are crucial to accurately forecast tax bases for corporate and personal income taxes. Finally, growth in employment and unemployment were considered the most relevant indicators for the labour market. As in previous post-mortem analyses, it was decided to define the outcomes used to assess the quality of the forecasts as the first available estimates of the national accounts. Forecast errors are defined as the difference between forecasts and realisations.

The quality of the annual forecasts is evaluated for three successive forecasting rounds, namely the yearahead forecasts in September and the current-year forecasts in February and September, with the emphasis being put on the first two rounds. The statistical tests performed give the following results. Forecast errors for all variables decline as the horizon shortens, but year-ahead forecasts exhibit sizeable mean absolute errors and contain little information on the variation around the sample mean. Errors are particularly large in the case of severe economic downturns or sudden changes in inflation. The positive bias observed in first-round real GDP forecasts is nonetheless not statistically significant and disappears completely in February. However, the GDP deflator suffers from a positive bias that is significantly different from zero in all forecasting rounds, which leads to an overprediction of nominal GDP. Gross operating surplus is the income component that captures the largest part of this overestimation. Employment growth forecasts are on average too pessimistic in all three rounds.

Other properties of the forecast errors are also examined. We find little evidence of persistence in the forecast errors. The February forecasts undeniably outperform naïve forecasts and have an improved accuracy for economic growth and inflation compared to the European Commission autumn forecasts. Growth accelerations and decelerations are anticipated in most cases in all the rounds. Lastly, forecast revisions tend to be too smooth, but this is a typical characteristic of macroeconomic forecasts. Overall, the forecasts of most of the variables considered in this analysis display the desired properties, but forecasts for real public consumption, the GDP deflator, the GFCF deflator and gross operating surplus clearly fall behind. Moreover, the analysis of the forecast errors in comparison with the results obtained in the previous study shows that the accuracy of the forecasts has generally improved over the last six years compared to the historical average. Yet, the reduction in the error size – in particular for the variables in real terms – is partly attributable to a decline in volatility of the outcomes.

Finally, we analyse to what extent the observed forecast errors in economic growth and inflation are due to errors made on the exogenous assumptions related to the international environment. Regression analyses show that if the evolution of potential exports markets had been correctly anticipated, the average absolute size of the year-ahead error on GDP growth would have been reduced by more than 60%. With a similar approach correcting CPI growth for errors made on import price developments, the mean absolute error of the year-ahead forecasts is decreased by 35%.