I’m a bit perplexed at a Gillian Tett piece in the Financial Times in which she is shocked, shocked that managers didn’t heed warnings from subordinates that risk models weren’t all that they were cracked up to be. Her article wouldn’t seem odd, except it focuses on a really basic shortcoming, namely that many models (Black Scholes, Value at Risk being some of the best known) assume a normal distribution of risk (also known as Gaussian or a bell curve). Anyone who knows the basics is aware that markets deviate from a normal distribution: they exhibit skewness (results are not symmetrically distributed around the mean) and “fat tails” or kurtosis risk (extreme events are far more probable than in a normal distribution).
Yet Tett reports that telling, or more accurately, reminding management of these failings is a career-limiting move:
A few years ago, Ron den Braber, an outspoken Dutch mathematics geek, was working in the risk department at Royal Bank of Scotland when he became alarmed about the models being used to price collateralised debt obligations.
Most notably, he concluded that the so-called Gaussian Copula approach then in use at RBS (and many other banks) significantly underplayed risks attached to the most senior pieces of debt – creating a danger of future, large losses.
So he duly tried to raise the alarm. But, as he tells the tale, he faced hostility. “I started saying things gently – in banks you don’t use the word ‘error’, but the problem is that in banks . . . people just don’t want to listen to bad news,” Mr den Braber recalls.
Now, every corporate tale has many sides – and RBS, for its part, vehemently denies that it ever ignores challenges or stifles debate. It says it could not find any record of strong warnings about the Gaussian Copula model, is aware of its shortcomings, and, while it has recently suffered CDO losses, these relate to products acquired after Mr den Braber’s time…
Or as one senior risk manager writes (anonymously since he remains employed): “[My] institution has now taken multibillion writedowns – job losses result and significant share price erosion – and I wonder how this can have happened? Upfront we did express to senior management that we lacked the analytical skills . . . and highlighted deep concerns about the approach colleagues in the market risk area had taken . . . I feel responsible for not doing more, but I really did push my views, risking my immediate career.”
Yves here. The second example, although less specific, is more troubling. Misplaced faith in analytical models is more understandable than handing risk management responsibility to a team that tells management is it not up to the task.
Back to Tett:
But, if nothing else, this saga shows the great blind spot that still haunts many banks. This decade, financiers have invented so many brilliantly clever mathematical tools to repackage risk that the industry has slipped, almost unthinkingly, into an assumption that “credit” is a collection of abstract equations, stripped from any human context.
Thus banks have become so dazzled with their powers that they have ignored how they interact with the rest of society – or how the tribal aspects of their own institutions can create dangerous traps.
Meanwhile, the cult of models has become so extreme that banks have believed them even when this collides with common sense. Yet, as any Latin scholar knows, the word “credit” hails from credere: “to trust”. It is, in other words, also a social construct.
And bankers forget this human dimension to their cost – no matter how impressive the abstract numbers might seem. Or as the same risk officer says: “The billions involved were so hard to contemplate that we almost certainly lost sight of the possible consequences [of our credit business] until it was too late.”
So, as the banks nurse their credit losses, they certainly do need to review why some of their clever mathematical models failed. That geeky Gaussian Copula stuff, in other words, matters hugely.
But, most important of all, they need to work out why the human processes around the models failed, too.But, if nothing else, this saga shows the great blind spot that still haunts many banks. This decade, financiers have invented so many brilliantly clever mathematical tools to repackage risk that the industry has slipped, almost unthinkingly, into an assumption that “credit” is a collection of abstract equations, stripped from any human context.
Thus banks have become so dazzled with their powers that they have ignored how they interact with the rest of society – or how the tribal aspects of their own institutions can create dangerous traps.
Meanwhile, the cult of models has become so extreme that banks have believed them even when this collides with common sense. Yet, as any Latin scholar knows, the word “credit” hails from credere: “to trust”. It is, in other words, also a social construct.
And bankers forget this human dimension to their cost – no matter how impressive the abstract numbers might seem. Or as the same risk officer says: “The billions involved were so hard to contemplate that we almost certainly lost sight of the possible consequences [of our credit business] until it was too late.”
So, as the banks nurse their credit losses, they certainly do need to review why some of their clever mathematical models failed. That geeky Gaussian Copula stuff, in other words, matters hugely.
But, most important of all, they need to work out why the human processes around the models failed, too.
Tett is on to something that a lot of professionals in banking no longer want to hear: credit worthiness depends on character as well as ability to pay. But assessment of character is subjective, and somehow institutions are not only reluctant to make assessments on a case-by-case basis, but distrust qualitative analysis.
One of the oddities of the banking industry is that despite all the talk of economies of scale, it’s utter rubbish. In the US, banks above a certain threshold (different studies draw the line in different places, but all come to the same conclusion) banks show a slightly increasing cost curve, meaning big banks are more costly to operate per dollar of assets, despite considerable cost efficiencies in certain areas (transaction processing, access to interbank funding). My pet, unproven view is that smaller banks know their communities better (is the hardware store a good business?) and make greater use of old-fashioned credit processes and that in the end, they are no more costly than quantitative, multi-level credit review processes (but if a big bank tried to revert to old-style lending, it might impose more costs for the same procedure than a small bank because it would have more portfolio/supervisory reviews).
Another FT writer, John Dizard, had a more cynical take on why financial firms continue to rely on demonstrably flawed Gaussian models:
As is customary, the risk managers were well-prepared for the previous war. For 20 years numerate investors have been complaining about measurements of portfolio risk that use the Gaussian distribution, or bell curve. Every four or five years, they are told, their portfolios suffer from a once-in-50-years event. Something is off here.
Models based on the Gaussian distribution are a pretty good way of managing day-to-day trading positions since, from one day to the next, risks will tend to be normally distributed. Also, they give a simple, one-number measure of risk, which makes it easier for the traders’ managers to make decisions.
The “tails risk” ….becomes significant over longer periods of time. Traders who maintain good liquidity and fast reaction times can handle tails risk….Everyone has known, or should have known, this for a long time. There are terabytes of professional journal articles on how to measure and deal with tails risk….
A once-in-10-years-comet- wiping-out-the-dinosaurs disaster is a problem for the investor, not the manager-mammal who collects his compensation annually, in cash, thank you. He has what they call a “résumé put”, not a term you will find in offering memoranda, and nine years of bonuses.
21 comments
Comments are closed.
Two points:
First, with regard to Tett, it would be interesting to look at the financial press over the past decade to see what coverage was given to these warnings as a percent of overall ink.
Second, Yves’ point about the costs associated with evaluating character is essential to ponder. Where we find higher cost, we also find limits to volume and speed. Once the larger financial firms chose to compete on volume and speed, attention to character was doomed. As we know, with regard to residential lending, the attention to character got outsourced — but unfortunately to the wrong players: mortgage brokers who fell in line with attention to volume and speed.
From an industrial market structure standpoint, any future effort to retain the advantages of volume and speed while not falling victim to inattention to slower, higher cost character determination will turn on finding a different set of players who have the skills, know how and dedication to character based evaluation.
They exist and are called non-profit affordable housing groups (the best ones anyway).
And, not only do the best ones have the skills and track record by the way (their delinquency and foreclosure rates are far smaller because they DO pay attention to character). But, in addition, they have a built-in set of motivations that sustain their competitive advantage: namely, they actually care about whether the borrower stays in the home. (Again, this refers to the best in class and certainly does not refer to ersatz firms who call themselves nonprofits but are close to being scams).
So, there’s a viable industrial approach out there. But, we’re unlikely to seize it because of cultural misunderstanding of non-profits ranging from naivete to arrogance.
Gaussian models are not the problem.
All banks with any type of model “sophistication” also have a suite of economic / financial shock “scenarios” for which they attempt to model risk. These are fat tail outcomes by intention and construction. Even this type of model becomes standardized and pretty much ignored by management when times are good.
So the problem is certainly not “Gaussian” distributions. The solution can’t be defined according to the confines and specifications of statistical or probabilty modeling. That simply casts the solution as yet another problem in the same category.
The problem is simply lack of common sense and good business judgement.
Well obviously the root cause is a lack of good business judgment but using the wrong mathematical models are a core piece of that judgment.
It isn’t just that they use Gaussian models — although it is a huge problem — in my opinion it’s that they view default risk strictly in terms of independent events instead of nonlinear dynamics. Even if they use a vastly different probability distribution that seems to explain the “chance” of something occurring, they would still make dumb decisions by declaring certain assets as uncorrelated with other assets — which might be true normally but not when there is a systemic meltdown.
I think the key point is that under normal conditions defaults tend to be pretty random and independent (and a normal distribution isn’t all that bad) but once the system has moved to a “tail” then positive feedback loops start dominating and suddenly many asset classes are highly correlated and affect each other negatively. This bifurcation of behavior is very common in nonlinear systems.
Gaussian models are not the problem.
This is correct. The problem is not with whether the model is normal or not. The problem is that many of the models assume that all (or most of) the random variables are independent, because it is just to hard to model the interdependencies otherwise. Even graphical stochastic models assume independence at each node.
This assumption is what results in the unexpected fat tails. If the variables are independent, then the probability is that any extreme event is “contained” (where have we heard that before) to that one variable. In reality, the variables are all dependent and so one extreme situation leads to another (though perhaps not immediately).
The issue with independence is just a modeling limitation well-known to statisticians. As anonymous remarked, the real issue is that the Wall Street types did not have enough common sense to know that the models could not handle everything.
If the big banks were indeed run by managers who did not understand the limits of their models of risk assessment , then one must then wonder what “model” was used to choose upper management. The one used did not insist that they have proper regard for the technical side of their bu8siness.
I would like to think that this applied just to banking. But the delays that both Boeing and Airbus are experiencing in production of their dream prducts suggest a similar disconnect there
And of course the greatest management failure of our generation the Rumsfeld Pentagon.
plschwartz
Sorry to say that any reference to “non-linear systems” as some sort of solution to the risk management problem makes me want to scream. It just begs the question. Another quantitative, detail burdened siren call.
Successful CEOs of banks that managed to avoid sub-prime and CDOs did not do so because of access to non-linear systems analysis.
I do think there’s something to be said for your pet theory that smaller banks no their communities better. But I have to disagree that any of this has to do with a failure of assessment of (a borrowers) character.
Lending institutions constructed their own house of cards where the way to finance their relaxed lending standards was to offer more credit. Debt piled on debt, leveraged to gross proportions. Character has little to do with it.
I may have missed it, but nowhere in the article or comments did I see any mention of capacity or ability to pay. Many of the problems of the residential mortgage market seem traceable to the inability of the borrower to service his or her debt. The situation arose either through the use of no doc loans (liar loans) or outride fraud on the part of the originators. I don’t know of any risk management system that will function if a key input like this is awry. Garbage in, garbage out.
“A once-in-10-years-comet- wiping-out-the-dinosaurs disaster is a problem for the investor, not the manager-mammal who collects his compensation annually, in cash, thank you. He has what they call a “résumé put”, not a term you will find in offering memoranda, and nine years of bonuses.”
When the bankers are handling OPM -(Other Peoples Money), they can hide behind the models, roll the dice and say crap like this “[My] institution has now taken multibillion writedowns – job losses result and significant share price erosion – and I wonder how this can have happened?”
Basically investors be damned, we got ours and let’s move forward.
I’ll bet Jamie Dimon, who knows what is going on in his business, didn’t listen to his modelers. He obviously didn’t need to “keep dancing” like his buddy C. Prince at Citicorp
Excellent piece addressing for example the need for relationships when lending and first hand looks at
the credit and the abstract look at modeling. Comments good too. In the modeling world a look at “correlation risk in a finite universe” could expose how large institutions ignoring correlation risk would naturally succumb to it the larger their portion of the universe’ correlated risk.
Still one has to wonder if these rational discussions really reach the root of the problem which appears to be that banking with solely a profit motive is now devoid of being beneficial socially (as it perhaps was more so in the past) and as such has no rational underpinning or rational structure vis a vis management, trading, models and risk. Which is to say, rather than the institutions being set up to employ theory to reduce risk, all the theory is just part of a facade to cover-up and rationalize rapacious short term greed and dishonesty with the consequent blindness to consequences both to society and the institutions themselves.
A model is just a tool anyway and believing in a model is naive –why brokers have long careers and traders don’t. Another example of positivism at its best,
believing one can mathematize nature and in so doing losing sight of nature itself.
“…in many cases the attempts to raise the alarm were crushed.”
the same thing going on at the banks was happening inside the halls of the US banking regulatory agencies. warnings were issued by a small number of analysts/economists about the growing re bubble about to burst and the lack of robust risk management programs within the banks. guess what, most of these voices were silenced. perhaps someday, someone will ask the regulators what they were doing while all of the risks were being put on the balance sheet instead of giving them broader powers.
ANON at 1:27,
it is not as Black-and-white.
Even Jamie Dimon had to concede that JPM has some problems with brokered loans and that they raised loan standards 6 times in 2007.
PLease read his excellent management letter in the AR for 2007.
I am not so sure that JPM is in great shape. Maybe relative to Citicorp – but that does not mean much.
What to conclude? Even super-smart guys like Dimon cannot move totally away from a bubble-way of doing business. Competitive pressures are too big; you have to maintain some market share; so you share some of the stupidities, even if you avoid many.
How does improving long-term risk metrics help a sales manager make next quarter’s bonus? How does it help improving the year-end stock price?
Management can make money on their options if the price goes up, walk away or even get repriced options if the stock goes down. For a manager to care about 10-year credit losses assumes that the manager is still going to be working at the same place in ten years, which is dubious.
Edmund Freeman,
Your comment gets at one of my pet peeves: the wreckage that has been created by letting investment banks go public. When partners were playing with their own money, they were vastly more cautious (although firms still managed to get themselves in trouble, witness First Boston, which was eventually taken over by Credit Suisse, or Lehman Brothers, which did itself a tremendous amount of damage due to infighting when the driving force of the fractious firm, Bobbie Lehman, died unexpectedly, or John Gutrfreund’s failure to call the Fed immediately when he learned a senior trader had been gaming Treasury auctions). However, comp was also kept within reasonable bounds, for the most part, because making partner was the big reward. Thus top talent had a reason to stay in place and build the firm. Incentives were vastly better aligned.
In commercial banks historically, there was much less turnover and promotions were based significantly on seniority. That had the effect of not encouraging risky behavior (and traditionally, banks were conservative and stodgy). If you did something risky and it paid off, you wouldn’t get paid more or promoted faster; indeed, you were likely to get a black mark for flouting the rules. But commercial banks decided they wanted to become investment banks and started changing their practices accordingly.
Purporting to put faith in wrong-headed mathematical models for careerist reasons: rather like Global Warming, then?
I don’t think this refusal to hear bad news is limited to the financial sector. I used to do promotion modeling and business forecasting for a large computer manufacturer. When the models we built and used did not create the answer management wanted, we changed the model and the assumptions rather than recognize that the problem might have been more intrinsic to the business instead.
Managers will always go with the answer they want, rather than the one that might be true.
There was a good article in Ethix magazine a couple of months ago about a local Seattle-based bank, Homestreet Bank, that made the decision to stick with FHA loans and not offer any subprime loans.
Today their bank is doing fine. They are not publicly traded.
http://ethix.org/article.php3?id=396
Some thoughts.
1/ As mentioned by some the problem is not really that the models asume a Gaussian distribution. Risk management in most firms proceeds by looking at V.A.R. numbers and risk scenario numbers. The risk scenarios calculate what happens (how much the firm earns or loses) when a tail event occurs.
The problem is that it is difficult to calibrate what extreme events are viable and need to be considered. If the stock market went down 95% is a day, every IB would collapse. If the housing and commercial real estate markets went down 40% in a day, pretty much every bank would collapse. These, however, are not considered viable extreme events, and no one considers them.
I have no expertise in housing or derivatives based on housing, but I imagine the banks are being faced with a market much like the one where housing goes down 40% in a day. It’s not doing that, of course, – it’s going down (say) 40% or so over the course of 5 or so years, but it’s *like* 40% in a day because the market for housing products has turned illiquid and no one is able to offload their products. Think of it in terms of stocks. Banks don’t collapse when the market goes down 5% because they suppose that when it does go down 5%, they can at least sell their shares and get out, thereby limiting their losses. But if stocks were to go down 5% every day for a while, *and* at the same time stocks became illiquid so that no one could sell their shares, then most IBs would indeed eventually collapse, because the cumulative effect would (eventually) be -95%.
So I imagine what caught the risk managers of housing products, is that the risk scenario which would have captured the market that we have fallen into (-40% or so on housing overnight, equivalent to a lengthy, more spread-out downturn of -40% *with* illiquidity of the products) was considered too extreme and just not a viable possibility. “Housing never falls 40% in a day! It goes down slowly!” The illiquidity may seem an obvious oversight, but then things are always more obvious post hoc.
2/ You can’t argue with success. This is by and large true, and does not hold just for financial firms. Greenspan was hailed as a maestro for a long time because the economy seemed to be doing so well. The counter-argument that some (including myself, and I imagine Yves) were making that things were doing well *now* but it would all end in tears, carries absolutely no traction until it actually does end in tears. American auto companies were doing very well with SUVs, invested massively, and now look ready to collapse. In financial firms there were (and always will be) people, including risk managers, who argue that the money being made today would be offset by losses down the road. But the certainty that money is being made now simply cannot be offset by the possibility of money, however much, might be lost in the future, because it only remains a possibility and is thus intangible.
No amount of modeling can replace plain old common sense. My credit union tells me they won’t have any sub-prime related losses, and I believe them. The managers have been avoiding that garbage from the beginning, and not necessarily because they’re experts on the “kurtosis risk” of Gaussian models.
I work for a relatively small bank, and I would love to think we make better lending decisions than bigger banks. However, to evaluate customer character you have to have customer contact, and even in a small bank the contact is made through a loan officer who needs to make loans to keep his or her job and/or who doesn’t have the experience to tell good character from bad. Post mortems on problem loans reveal with depressing frequency that problems were swept under the rug by the loan officer at the origination stage. Agency problems at the loan officer level are the same regardless of the size of the bank.
I am always surprised at blaming the models.
First of all models are guides and frameworks with limitations. Its a guide and not a decision-tree.
Second you follow lending principles and resultant loan book looks like Gaussian Distribution. You cannot aim for Gaussian distribution – for that process in itself creates a skew.
Thirdly, the incentives are all misaligned – agree with Yves here.
Fourth – banking is not telecom where you go for 100% coverage – banking by definition excludes a certain portion of population by defining them as unbankable. One needs to cross a threshold to be able to bank. And this threshold cannot go lower indefinitely.
Combine all – and you have Subprime!
Rahul