Roland Dohrn
1. Background and Structure of the Paper
Evaluating forecasts is an exercise which is done more or less regularly by pro-
ducers of business cycle forecasts as well as independent researchers. Re-
viewing recent literature on this issue, the papers aim at quite different di-
rections. Some primarily take the users’ view and present implicitly a ranking
of different forecasts with respect to their accuracy (e.g. Oller, Barot 2000).
Other analyses try to detect common features from various forecasts by dif-
ferent institutions (Rouss, Savioz 2002; Dopke, Fritzsche 2005). A different
strand of research comes from institutions evaluating the accuracy of their
own forecasts often employing a wide spectrum of statistical measures
(Keereman 1999; Kontsogeorgopoulos 2000; Timmermann 2006). Finally, only
few researchers try to gather hints from past forecast errors, how to improve
future forecasts (for Germany e.g. Neumann, Buscher 1985; Kirchgassner,
Savioz 2001; for the U.S. Steckler 2002:232-233 gives an overview).
This paper tries to add some new evidence to the last category from a fore-
caster’s perspective. For that purpose, some short term forecasts for the
German economy the author of the paper is engaged in are analysed. In par-
ticular it is asked whether a broad set of information available at the time of
the projection has been used properly. This issue is addressed as one aspect of
informational efficiency of forecasts in the forecast evaluation literature.
Various tests for informational efficiency are proposed having one feature in
common: They analyse forecast errors with respect to their correlation with
past data supposed to contain information about future economic devel-
opments and presumably have been known to the forecaster when he made
his prediction. If errors are uncorrelated with such variables, it can be assumed
that the accuracy of the forecast could not have been improved by making a
better use of these data. If the opposite is true, there should be reason to check,
whether the data in question could have been utilised better.
As far as this kind of analyses has been carried out earlier the focus mostly was
on GDP forecasts. However, it must be taken into account how short term
forecasts are made in practice. Without going into detail, they are mostly built
bottom-up, starting with forecast of the expenditure side of GDP at a
disaggregated level. These detailed forecasts are added up to yield a GDP
forecast, which, however, may be subject to some fine tuning thereafter.
Therefore it would help little to improve predictions if a correlation is de-
tected between some economic indicators and the errors of GDP forecasts. If,
e.g., it turns out that the error in GDP forecast and share prices are correlated,
a forecaster would not know whether he failed in assessing the wealth effect,
which would require to scrutinize consumption forecast, or whether he did not
take into account properly the impact of share prices on financing conditions
and, thus, on investment. Therefore, this paper also takes into account the in-