Richard Alford: Why Economists Have No Shame – Undue Confidence, False Precision, Risk and Monetary Policy

By Richard Alford, a former New York Fed economist. Since then, he has worked in the financial industry as a trading floor economist and strategist on both the sell side and the buy side.

In theory, there is no difference between theory and practice. In practice, there is. – Yogi Berra

Economic policymakers, pundits and academics continue to forecast the future course of the economy and predict the effects of possible policy initiatives with an air of scientific certainty. The high degree of confidence expressed in their forecasts, predictions and commentary continues unabated despite:

1. Only a small minority of economists and none of the central banks and treasury/finance ministries anticipated the financial crisis and the recession, and

2. At most only one of the currently competing macroeconomic models (which embody significantly different structures and implications for economic policy) can serve as a sound basis for policy.

The confidence and false precision in these forecasts and policy prescriptions reflect a continuing unwarranted faith in the models. With Greenspan’s defense, “no one saw it coming” and Bernanke’s admission that monetary policy is no panacea still fresh, some economists, including Brad DeLong, are now asserting that they “have reason to be proud of our analyses over the last five years.” This assertion flies in the face of the fact that macroeconomic policymakers and both “freshwater” and “saltwater” economists had embraced a model (Dynamic Stochastic General Equilibrium) that assumed away the importance/existence of a financial sector. This recasting of recent history would make Orwell proud.

The undiminished confidence of mainstream economists also raises a number of questions:

1. Why do economists, policymakers and pundits continue to express high degrees of confidence when forecasting or commenting on the merits of alternative policy prescriptions given their recent dramatic failure to forecast the crisis and recession and their widely divergent models?

2. Does the dynamism and complexity of the economy allow for a model(s) that can predict the future course of the economy and the efficacy of macroeconomic policy with the precision and degree of confidence that these economists and policymakers assert?

Undue Confidence and False Precision

Bernanke’s statement that monetary policy is no panacea reflects the degree to which economists, policymakers, and politicians have oversold the ability of economics and economic policy to deliver satisfactory economic performance. Ironically, no groups have been bigger buyers of the ability of macroeconomic policy to ensure satisfactory, if not optimal, outcomes than macroeconomists and policymakers themselves.

The majority of mainstream economists and policymakers failed to see, comprehend and react to the imbalances and unsustainabilities that became integral parts of the calm before the storm (aka the “Great Moderation”) and that contributed to the severity of the financial crisis and the recession. They acted and set policy as if:

1. Any projected path for the economy that was inconsistent with their model was out of the realm of possibility and therefore irrelevant, and

2. Any divergence between the model’s predictions and the real economy would be corrected by the real economy moving back into line with the model.

This suggests that macroeconomists have committed an error of misplaced concreteness, i.e., they confuse their models of the economy with the real economy itself. This confusion of the model with the real economy is hazardous. In 2010, Ricardo Caballero of MIT put it this way:

… the so-called dynamic stochastic general equilibrium approach—has become so mesmerized with its own internal logic that it has begun to confuse the precision it has achieved about its own world with the precision that it has about the real one.

Caballero also noted:

…This is dangerous for both methodological and policy reasons. On the methodology front, macroeconomic research has been in “fine-tuning” mode within the local-maximum of the dynamic stochastic general equilibrium world, when we should be in “broad-exploration” mode. We are too far from absolute truth to be so specialized and to make the kind of confident quantitative claims that often emerge from the core. On the policy front, this confused precision creates the illusion that a minor adjustment in the standard policy framework will prevent future crises, and by doing so it leaves us overly exposed to the new and unexpected.

However, since the financial crisis and the recession, there has been very little in the way of a shakeup in the economics profession. True to form, the economists are pursuing only marginal changes in their models. Mainstream defenders of the current paradigm argue that the crisis and the recession were the result of regulatory failures alone and that macroeconomic models and policy are blameless and do not require significant alteration. Non-mainstream economists find fault in the reigning model and seek to replace it with a model of their own design.

The differing camps in the debates have deeply held world views and different models. Each camp claims that their findings are truly scientific and that anyone who disagrees with them must be mentally deficient, ethically challenged, and genetically unsound. Each camp has a shared economic model and places great faith in the ability of the model to predict, explain and serve as a policy tool. However, despite all the disagreement and name calling, the economists engaged in the debates share one fundamental position: economics is a science and potentially of great utility if their model is employed.

In recent debates about the fiscal stimulus, numerous economists and pundits opined with high degrees of confidence on the effectiveness of fiscal stimulus. The statements were either implicitly or explicitly based on an assumed value for the fiscal multiplier. Given that the fiscal multiplier has been around since Keynes wrote “The General Theory…” one might imagine that a consensus about the value of the fiscal multiplier has emerged. However, a CBO report titled “Estimated Impact of the American Recovery and Reinvestment Act (ARRA) on Employment and Economic Output from October 2011 Through December 2011” (February 2012) pointed out:

A recent survey of studies based on historical evidence shows that estimates of fiscal multipliers range from -0.3 to 3.6, although most of the estimates fall between 0.5 and 2.0. Several of those studies yield estimates that average between 0.5 and 1.0 over a long historical period. However, those studies do not specifically provide estimates for a period in which unemployment is high and interest rates are very low, as is true for the current period.

The range of estimated values of the multiplier based on the historical record is inconsistent with the confidence expressed in many public statements about the impact of fiscal stimulus. Ex post efforts at quantifying the impact of the ARRA also show a wide range of values for the implied multipliers. For example, the CBO’s report on the effects of the ARRA (actual Q4 2011 compared with what would have occurred in the absence of the ARRA) is as follows:

1. It raised real (inflation-adjusted) gross domestic product (GDP) between 0.2 and 1.5 percent,

2. It lowered the unemployment rate between 0.2 and 1.1 percentage points,

3. It increased the number of people employed between 0.3 and 2.0 million, and

4. It increased the number of full-time-equivalent jobs by 0.4 to 2.6 million.

The estimates at the high end of the CBO’s range of ARRA’s effect on real GDP growth, the number of people employed, and the increased number of FTEs are about 7x the size of the low end of the range. Keep in mind that these are ex post efforts at quantification and not the less certain ex ante forecasts. The range, 7x the size of the low estimate, is also considerably larger than the 4x range in the CBO’s reported central tendency of economists’ estimates of the value of the multiplier.

The sizes of the ranges are inconsistent with the apparent degree of confidence that individual economists and camps of economists place in their models and forecasts. Recent research may help to explain the confidence placed in forecasts. A paper by Soyer and Hogarth shows that the standard method of presenting regression results in the economic literature demonstrates an “illusion of predictability,” i.e., statistically significant average effects are emphasized at the expense of the distribution of outcomes:

… the way in which results are presented in regression analyses obfuscates the uncertainty inherent in the dependent variable.

Soyer and Hogarth couch their result in a policy framework:

… consider a decision maker who is pondering which actions to take and how much to do so in order to reach a certain goal. This requires conjectures to be formed individual outcomes that would result from specific inputs. Moreover, the answers to these questions depend not only on estimating average effects, but also on the distribution of possible effects around the average.

US monetary policy has been guided by the Taylor Rule. The Taylor Rule is a decision rule designed to minimize the loss to society from deviations of inflation and unemployment from target, given estimated parameters and weights. The distribution of potential outcomes is not reflected in the Taylor rule.

The Limits of Economics as a Predictive Science

Well into 2007, economists, pundits and policymakers were confidently predicting a continuation of the Great Moderation, and many were issuing assurances that there was no housing price bubble. Does this failure reflect problems with economists, policymakers and their models? Or is the problem inherent in the nature of the economy? If the failure of mainstream models to foresee the crisis and recession is the result of economists having too much faith in a faulty model, then changing the economists or the model could rectify the problem. If, on the other hand, the nature of the macroeconomic problem is such that models that allow for forecasts and policy design with the required accuracy are impossible, than how should policy be designed, implemented, and communicated?

While many economists would argue that economics is on a solid scientific foundation, others would disagree. Caballero of MIT quotes von Hayek:

…In his Nobel-prize acceptance lecture, Hayek writes: “Of course, compared with the precise predictions we have learnt to expect in the physical sciences, this sort of mere pattern predictions is a second best with which one does not like to have to be content. Yet the danger of which I want to warn is precisely the belief that in order to have a claim to be accepted as scientific it is necessary to achieve more. This way lies charlatanism and worse. To act on the belief that we possess the knowledge and the power which enable us to shape the process of society entirely to our liking, knowledge which in fact we do not possess, is likely to make us do much harm” (von Hayek, 1974).

Karl Popper and other philosophers of science would also argue that economics, among other social sciences, would never exhibit the explanatory and predictive power of a hard science, e.g., physics:

Long-term prophecies can be derived from scientific conditional predictions only if they to systems which can be described as well isolated, stationary, and recurrent. These systems are very rare in nature; and modern society is not one of them.

Economists have long been aware that the structure and behavior of an economy is not stationary, but rather evolves over time and in response to changes in policy regimes. This is reflected in the Lucas Critique, which links adjustments in economic policy to changes in the decision-making rules employed by economic agents and hence the behavior of the economy. Goodhart’s Law states the economic policy dimension of the problem succinctly:

Any observed statistical regularity will tend to collapse once pressure is placed upon it for control purposes.

Put another way, when policymakers implement policy based on a statistically identified past pattern of behavior of economic agents, that behavior will change. As a result, the economy will not evolve or respond to the policy as predicted. The existence of feedback loops is not limited to macroeconomics. It has been noted elsewhere in the social sciences, i.e., Campbell’s Law:

The more any quantitative social indicator is used for social decision-making, the more subject it will be to corruption pressures and the more apt it will be to distort and corrupt the social processes it is intended to monitor.

If economic policymakers had considered von Hayek, Popper, the Lucas Critique, Goodhart’s Law, or Campbell’s Law, then they would not have been overly confident in the ability of the announced low-for-long interest rate policy and the tightening at a constant, well-telegraphed “measured pace” to produce either stable growth or financial stability. Furthermore, they would have been skeptical of the regulator-imposed Value-at-Risk system’s ability to measure and control risk in markets.

This does not imply that modeling is a dead end. Caballero again:

…There is no doubt that the formalization of macroeconomics over recent decades has increased its potential. We just need to be careful to not let this formalization gain its own life and distract us from the ultimate goal, which is to understand the mechanisms that drive the real economy.

The world is dynamic and economic agents adapt quickly to change if they have not already anticipated it. Consequently, forecasters and policymakers must deal with uncertainty despite the fact that economic agents in general want certainty and that economic models must be tractable in order to be useful.

Policy Risk and Policy Risk Management

In a 1967 American Economic Review article, Brainard showed that uncertainties about the state of the world and the effectiveness of policy imply that economic policy is risky. The riskiness implies that policymakers should take into account not only the most likely effect of the policy as predicted by the underlying model, but the range of possible outcomes as well.

This in turn points to the need for a policy risk management discipline. The idea of risk-adjusted policy may appear novel and non-operational, but it is not. In the mid-1990s, US monetary policy reflected an effort to incorporate policy risk management as it dealt with the implications of forecast errors and policy lags. In “Central Banking Theory and Practice,” (MIT Press,1998), former Vice Chairman of the Board of Governors of the Federal Reserve System Alan Blinder shed light on the Fed’s procedure given the desire to move preemptively despite the uncertainty/risk:

A preemptive strategy implies a certain amount of confidence in both your forecast and your model of how monetary policy affects the economy, both of which are hazardous. But preemption does not require too much confidence. Remember the flexibility principle of dynamic programming and the Brainard conservation principle. Taken together, they led to the following sort of strategy:

Step 1- Estimate how much you need to tighten or loosen policy to “get it right.” Then do less.

Step 2- Watch developments.

Step 3a -If things work out about as expected, increase your tightening or loosening to where you thought it should be in the first place.

Step 3b- If the economy seems to be evolving differently from what you expected, adjust policy accordingly.

Policy can be appropriate. It can also be or become inappropriate. Policy uncertainty precludes “Set it and forget it!” policy design as well as implementation. The underlying uncertainty necessitates a certain tentativeness and constant monitoring of economic developments by policymakers in order to gauge whether or not the economy is evolving as anticipated. Given the Lucas Critique and Goodhart’s Law, the policymakers must also have open minds and actively look for “the new and unexpected.”

The Fed practiced a form of policy risk management in the mid-1990s. However, post the NASDQ bubble the Fed set monetary policy based on expected returns only and ignored the new and unexpected. It now appears that the FOMC is again considering the risks as well as the expected return as it ponders additional unconventional policy steps. The uncertainty and risks attached to additional policy steps are currently high relative to the expected returns. The economy, the financial markets and policy are in unchartered waters and the unconventional policy steps are not as effective as conventional policy was in affecting the real economy. Given the need to adopt a policy risk management regime in the mid-1990s, the current heightened uncertainties, and the relatively low expected returns to further unconventional steps, one should not be surprised by the FOMC’s recent reluctance to take additional steps.

Economists, policymakers, portfolio and asset managers face tradeoffs between risks and returns. Asset and portfolio managers should be called to task if they ignore risks and focused solely on expected return. So should policymakers. Macroeconomic modelers and policymakers might consider holding themselves to a standard similar to the standard suggested for financial modelers by Paul Wilmott and Emanuel Derman in their “The Financial Modelers’ Manifesto,” e.g.:

The Modelers’ Hippocratic Oath

~ I will remember that I didn’t make the world, and it doesn’t satisfy my equations.

~ Though I will use models boldly to estimate value, I will not be overly impressed by mathematics.

~ I will never sacrifice reality for elegance without explaining why I have done so.

~ Nor will I give the people who use my model false comfort about its accuracy. Instead, I will make explicit its assumptions and oversights.

~ I understand that my work may have enormous effects on society and the economy, many of them beyond my comprehension.

_________________

Blinder, Alan, Central Banking in Theory and Practice, MIT Press, 1998, page 17.
Brainard, William, “Uncertainty and the Effectiveness of Policy”, AER, vol. 57, No. 2, (May 1967) pp.411-425.
Ricardo Caballero, Macroeconomics after the Crisis: Time to Deal with the Pretense-of-Knowledge Syndrome , http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1683617
CBO, Estimated Impact of the American Recovery and Reinvestment Act on Employment and Economic Output from October 2011 Through December 2011, http://www.cbo.gov/publication/43013
Emre Soyer and Robin Hogarth, The Illusion of Predictability: How Regression Statistics Mislead Experts, http://papers.ssrn.com/sol3/papers.cfm?abstract_id=1996568
Paul WIlmott and Emanuel Derman , “The Financial Modelers’ Manifesto”, :http://www.wilmott.com/blogs/eman/index.cfm/2009/1/8/The-Financial-Modelers-Manifesto