By Hites Ahir, Senior research officer, IMF, and Prakash Loungani, Senior resource manager and advisor in the IMF’s Research Department. Originally published at VoxEU.
Since the onset of the Great Recession, much of the world has been in a state of economic winter: nearly 50 countries were in recession in 2009 and 15 countries slipped into recession in 2012. After weak global growth in 2013, economic forecasters are predicting a rosier outlook this year and next. Can these forecasts be trusted? Or are forecasters simply intoning – like Chauncey Gardner, the character played by Peter Sellers in the movie Being There – that “there will be growth in the spring”?
The 2008–2012 Record
In a classic 1987 paper, William Nordhaus documented that, as forecasters, we tend to break the “bad news to ourselves slowly, taking too long to allow surprises to be incorporated into our forecasts.” Papers by Herman Stekler (1972) and Victor Zarnowitz (1986) found that forecasters had missed every turning point in the US economy.
These traits appear to have persisted to this day. In our recent work we look at the record of professional forecasters in predicting recessions over the period 2008-2012 (Ahir and Loungani 2014). There were a total of 88 recessions over this period, where a recession is defined as a year in which real GDP fell on a year-over-year basis in a given country. The distribution of recessions over the different years is shown in the left-hand panel of Figure 1.
Figure 1. Number of recessions predicted by September of the previous year
The panel on the right shows the number of cases in which forecasters predicted a fall in real GDP by September of the preceding year. These predictions come from Consensus Forecasts, which provides for each country the real GDP forecasts of a number of prominent economic analysts and reports the individual forecasts as well as simple statistics such as the mean (the consensus).
As shown above, none of the 62 recessions in 2008–09 was predicted as the previous year was drawing to a close. However, once the full realisation of the magnitude and breadth of the Great Recession became known, forecasters did predict by September 2009 that eight countries would be in recession in 2010, which turned out to be the right call in three of these cases. But the recessions in 2011–12 again came largely as a surprise to forecasters.
A Robust Finding
In short, the ability of forecasters to predict turning points appears limited. This finding holds up to a number of robustness checks (Loungani, Stekler, and Tamirisa 2013).
- First, lowering the bar on how far in advance the recession is predicted does not appreciably improve the ability to forecast turning points.
- Second, using a more precise definition of recessions based on quarterly data does not change the results.
- Third, the failure to predict turning points is not particular to the Great Recession but holds for earlier periods as well.
Figure 2. Predicting recessions in advanced economies, 1989–2008
These points are illustrated in Figure 2 above, which presents evidence on how well recessions were predicted in advanced economies over the period 1989 to 2008. Consider the panel on the left. The line labelled ‘unconditional forecast’ shows the normal evolution of forecasts of real GDP growth. On average across countries, forecasters start out by predicting – in January of the preceding year – that annual growth in the following year will be about 3%. This forecast is then lowered slightly over the coming 24 months.
The evolution of forecasts in years that will turn out to be recessions is shown by the red bars. While forecasts in recession years start out very close to the unconditional average, they start to depart from it slightly around the middle of the previous year, suggesting that forecasters are already starting to be aware that the year to come is likely to be a departure from the norm. Major departures of the forecasts from the unconditional average, however, only start to occur over the course of the current year and occur in a very smooth fashion. By the end of the forecasting horizon in December, forecasts are only slightly above the average outcome in recession years, which is shown by the solid blue line. (Note that, on average, growth is not negative during recessions in advanced economies because the dating of recession episodes is based on the quarterly data and annual growth tends to remains positive during many recessions.)
The evidence for recessions associated with banking crises is shown in the right panel of Figure 2. Here, the departure from the unconditional forecast starts earlier than it does for other recessions. This is followed by a smooth pattern of downward revisions to the forecast. In this case, even the terminal forecast greatly underestimates the actual decline, which on average is about 1.5%. To summarise, the evidence over the past two decades supports the view that “the record of failure to predict recessions is virtually unblemished,” as Loungani (2001) concluded based on the evidence of the 1990s.
Are Official Forecasts Any Better?
Forecasts from the official sector, either from national sources or international agencies, are no better at predicting turning points. In the case of the US, the March 2007 statement by then-Fed Chairman Bernanke that “the impact on the broader economy and financial markets of the problems in the subprime market seems likely to be contained” has received a lot of attention. Another Fed Chair, Alan Greenspan, told his colleagues in late-August 1990 – a month into a recession – that “those who argue that we are already in a recession are reasonably certain to be wrong.” Forecasts by Fed staff have also missed turning points, as discussed in Sinclair, Joutz and Stekler (2010).
During the Great Recession, Consensus forecasts and official sector forecasts were so similar that statistical horse races to assess which one did better end up in a photo-finish. As an example, Figure 3 compares Consensus forecasts with those made by the OECD. For both the year-ahead forecasts (those made in June of the previous year) and the current year forecasts, the correlation between the two sources of forecasts is extremely high. Hence, an analysis of the OECD’s ability to forecast recessions gives results very similar to those shown earlier for Consensus forecasts.
Figure 3. Correlation between Consensus forecasts and OECD forecasts
For the case of the IMF too, a recent independent evaluation concluded “that the accuracy of IMF short-term forecasts is comparable to that of private forecasts. Both tend to overpredict GDP growth significantly during regional or global recessions, as well as during crises in individual countries” (IMF 2014).
Searching for an Explanation
To paraphrase Oscar Wilde, to fail to forecast a few recessions may be misfortune, to fail to forecast nearly all of them seems like carelessness. Do forecasters simply not update their forecasts often enough to be alert to the onset of recessions? That simple possible explanation turns out not to be true. As shown in Figure 2 earlier, the consensus forecasts in recession years are revised every month; they just are not revised down enough to capture the onset of recessions. Related work by one of us – which looks at the behaviour of individual forecasters rather than just the consensus – also finds that forecasts are updated quite often (Dovern, Fritsche, Loungani, Tamirisa, 2014).
So the explanation for why recessions are not forecasted ahead of time lies in three other classes of theories, which are not mutually exclusive.
- One class says that forecasters do not have enough information to reliably call a recession. Economic models are not reliable enough to predict recessions, or recessions occur because of shocks (e.g. political crises) that are difficult to anticipate.
- A second class of theories says that forecasters do have not have the incentive to predict a recession, which – though not a tail – event are still relatively rare. Included in this class are explanations that rely on asymmetric loss functions: there may be greater loss – reputational and other kinds – for incorrectly calling a recession than benefits from correctly calling one.
- The third class stresses behavioural reasons for why forecasters hold on to their priors and only revise them slowly and insufficiently in response to incoming information (Nordhaus 1987).
Regardless of the explanation for why recessions fail to be forecasted, the evidence suggests that users of these forecasts need to be cognizant of this feature. Forecasters may be predicting that “there will be growth in the spring” but their ability to predict a sudden snowstorm is rather limited.
References
Ahir, Hites and Prakash Loungani (2014), “Fail Again? Fail Better? Forecasts by Economists During the Great Recession”, George Washington University Research Program in Forecasting Seminar.
Dovern, Jonas, Ulrich Fritsche, Prakash Loungani, and Natalia Tamirisa (2014), “Information Rigidities: Comparing Average and Individual Forecasts for a Large International Panel”, IMF Working Paper 14/31.
IMF, Independent Evaluation Office (2014), “IMF Forecasts: Process, Quality and Country Perspectives”.
Loungani, Prakash (2001), “How accurate are private sector forecasts? Cross-country evidence from consensus forecasts of output growth”, International Journal of Forecasting, 17(3).
Loungani, Prakash, Herman Stekler, and Natalia Tamirisa (2013), “Information rigidity in growth forecasts: Some cross-country evidence”, International Journal of Forecasting, 29(4).
Nordhaus, William (1987), “Forecasting Efficiency: Concepts and Applications”, Review of Economics and Statistics, 69(4).
Sinclair, Tara M, Fred Joutz, and Herman Stekler (2010), “Can the Fed predict the state of the economy?”, Economics Letters, 108(1).
Stekler, Herman (1972), “An analysis of turning point forecast errors”, American Economic Review, 62.
Zarnowitz, Victor (1986), “The Record and Improvability of Economic Forecasting,” NBER Working Paper 2099.
17 comments
Comments are closed.
Maybe we should consider the hypothesis that “forecaster” is a euphemism. Almost all of these so-called forecasters work for some enterprise that has an ongoing interest in their not being a recession, and their job is to disseminate economic information that helps advance their employers’ interests.
Asking why economic forecasters do not predict recessions is a bit like asking why psychiatric practitioners do not predict their own patients’ suicides.
Yes…this profession has morphed from “analytical evaluation” into public relations/promotion…this function needs to be carried out by completely unvested interest….the government, realtors, etc can’t be relied upon to provide unbiased assessments. They provide “hopeful” assessments, with the idea that their optimism will translate into consumer confidence….kind of reminds me of that “Green Shoots” thing a few weeks after the start of the GFC.
Almost everything about this paper is wrong. Lets start with its definition of recession as a year over year reduction in GDP. Even the NBER which officially, if retrospectively, calls recessions does not rely solely on GDP. And too the US “officially” exited the recession which began in December 2007 some 4 years ago. However, the reality is that for 80% of Americans that recession/depression continues.
The forecasters completely misunderstand the nature of forecasting. It is not about lame models. A lot of us knew, for instance, a couple years out that there was a housing bubble and that at some point it was going to pop. But there were simply too many factors and possible factors to tell when exactly it was going to blow up. The conditions can be in place for some time for a bubble to go splat, but there needs a precipitating event to act as the final pinprick. In the housing bubble, that was provided by the freezing up of some Paribas funds tied to mortgages in August of 2007. Once the bubble burst, it was not especially hard to see the strong likelihood of recession in the following months. But that forecast could have been belied if the Fed and government had acted aggressively to support homeowners and to unwind the much larger and more explosive CDO/CDS mess lying in wait. So part of the forecast was that they weren’t going to. Today we have an economy which is even more financialized. We have multiple bubbles in equities and commodities which need ongoing accommodative Fed policies, i.e. giveaways to sustain them. Very likely at some point even Fed policy won’t be enough for them, but Fed policies, the Greenspan-Bernanke-Yellen puts, have been able to keep these bubbles going for 4 years. Many of us expect another crash but the when of it as with the housing bubble which preceded it is anybody’s guess. Fed policy could keep all the balls in the air for several more years or conditions in Europe or China could suddenly go south and push us over the edge as well. The point is that all the conditions for the next crash are already in place and, while there is endless can kicking going on, nothing is being done to actually defuse this next crisis. That should be the essence of any reality-based forecast. Our economy is not healthy. Indeed our society is not. It is rigged with explosives. It is set to blow up and the forecast is it will blow up.
The Fed will not be able to keep the balls levitated much longer, because it uses other nations’ economies to bounce the balls off of – there is virtually zero sense within the US of what the crash and aftermath and Fed reflating/no regulating combined with a stance of unparalleled US hostility in so many areas around the globe (75 countries experience US sovereign intrusions for reasons of assassination, kidnapping, sabotage, regime change, corporate protection, drug operations, the works) has done to the global financial system, business climate, corporate investment etc., beyond what once was called the ‘First’ world.
When there is no area of promising policy even under discussion, no prospect via the partisan political route of a return to the ‘golden days’ of seemingly unlimited wealth and opportunity, an open attack by the elite and its captured State(s) on the weak in all subject populations, no tolerance for independent action of any kind by other large national/cultural players, along with active military aggression in a dozen countries currently minimum, the essential ingredient of stability has left the room.
In any event, the best forecasters we’ve ever produced are “thinkers” and science-fiction writers – the Fed will never see the one that kill’s it coming, as it pretty much counts beans.
This is hardly the first time this phenomenon has been studied. Here’s a link to an Economist’s View post, now nearly a decade old: http://economistsview.typepad.com/economistsview/2005/08/the_use_of_lead.html
Simply put, most “forecasters” are actually trend-followers. They project the existing trend forward. That’s why they miss turning points. It’s also why their record was better in 2009 once the existing recession was well known, and unlike most recessions, persisted for a very long time.
On the contrary, if you just Keep It Simple Stupid and make use of the index of Leading Economic Indicators, you will be right much more often than most pundits. But where is the money in that?
Further, economic forecasting can only be correct to the extent that the forecaster is the proverbial bug on the wall. Forecasters with known good records, like Joe Granville in the 1970s and 1980s, move the market with their opinions. Suppose there was a FED forecaster with a known perfect record, who predicted a recession 9 months from now. What would you (and CEO’s and everybody else) do? You’d position yourself for the recession now, probably by production cuts and layoffs. The forecast of the future would become a report of the present.
While I’m at it, let me say a few words about the yield curve post above this. While the yield curve inverted in 1928, it did not invert at all between 1932 and the early 1950s, and yet we had several recessions, including the very bad recession of 1938, in that era. My take on it is that an inverted yield curve is always bad, whereas a positive yield curve isn’t always good. That makes the UK data interesting, because they did have inverted yield curves, but they persisted through the recession into the early recovery. I suspect this is because the US is the dog, and Europe has been the tail. More interesting would be to compare 20th century US yield curves with 19th century UK yield curves (when they were the pre-eminent power).
LEI missed the GFC, if I remember correctly.
American-style economists are the 21st century priestly class. They are bound to a few tenets and exist to protect the masters and the appearance of the status quo. Changing forecasting models is anathema because it undermines their tenets and may cast their masters in a poor light, but in the end, economists receive patronage to prevent political change.
Does anything post here?
One reason it’s hard to forecast recessions in real time is that they aren’t officially dated by the NBER until months after the fact. Here is the NBER’s Sep. 2010 communique, announcing that the Great Recession ended in … June 2009. Wow, thanks, guys!
http://www.nber.org/cycles/sept2010.html
It would not be difficult to select four or five key time series and develop a public, quantitative, multi-factor definition of recession, which could be determined in near real-time with only a month or two delay for data publication.
If the NBER won’t do it, then I’ll just have to do myself. No more driving blind, waiting on a bunch of old farts to schedule a teleconference. I’m done with that crap, comrades.
That is one of the most refreshing and optimistic statements I’ve heard in a very, very long time.
haha, the best movie!
“Since the onset of the Great Recession, much of the world has been in a state of economic winter…”
Frameworks like this make me curious about what exactly commentators think started with the onset of the Great Recession? Is the implication that things were hunky-dory before this onset?
Flipping a coin would give the right answer half the time. How can economists justify their salaries? A science with no predictive value, is no science at all.
Today’s economists remind me of Chauncey Gardiner. It has to be frustrating because they have no firm figures of the debt Wall Street created. Some reports say as much as $600 TRILLION – some say even more. The unknown equations make it impossible to accurately predict. On top of the debt there is the Fed printing untold amounts of money buying bad debt with no accounting of where the money is actually being spent or used by the banks.
Investors are not coming back until there are some safeguards in place and legislators are not willing to risk their re-election funds to reinstate regulation. Although the banks have paid penalties, fines and costs (not nearly enough for the damage they have caused), no court has confiscated their patents or software systems yet – so it continues to be business as usual and getting caught is just a cost of doing business.
On a positive note, when you review JudicialWatch and OpenSecrets financial disclosure statements – politicians and federal judge’s have significantly unloaded bank stocks when you compare the years from 2007 – present.
Those practicing an inherently fraudulent discipline making predictions about something that is inherently unknowable. Of course it’s not going to work out well.
I’m willing to bet that if you took a poll of astrologer predictions about the future they’d be just as accurate as those from economists. And therefore since astrologers would certainly be cheaper to keep on the payroll you all ought to fire your economists and replace them with astrologers.
Chauncey Gardner might have stuck his neck out by predicting there will be growth in Spring. The best you’d get from any economist is that there may be growth in Spring.
As noted above, science-fiction writers and multidisciplinary “thinkers” have consistently produced far better projections of where American society or humanity as a whole is or may be headed, whereas in economics the select-for-orthodoxy system long in place has all but excluded the sort of imagination and creativity required to see a forest except in paint-by-number.