Bill Black: How the “Super Crunchers” Became the “Super Torturers” of Finance Data

Yves here. Many of the concerns about Big Data focus on the surveillance apparatus used to collect it, or on the naive modeling approaches, like attributing causality to mere correlations. Here Black addresses an established problem: that of deliberate abuse of models.

By Bill Black, the author of The Best Way to Rob a Bank is to Own One and an associate professor of economics and law at the University of Missouri-Kansas City. Originally published at New Economic Perspectives

Some books have spectacularly bad timing, like Moral Markets: The Critical Role of Values in the Economy, which was published in 2008 as a celebration of market and their nourishment of high ethical values.  The book has many interesting chapters and I recommend it, but even lifelong market apologists now refer to the “corrupt culture of banking.”  Ian Ayres, a brilliant professor of law and economics at Yale, published his book Super Crunchers: Why Thinking-By-Numbers is the New Way To Be Smart to critical acclaim on August 28, 2007.  Ayres’ book is an ode to how much better decision-making becomes when it is made empirically on the basis of very large data rather than through human judgment.  There is a great deal of support in the literature for that thesis, and the result is one of the reasons why behavioral economics has become increasingly dominant.  The general idea is that humans bring significant, unexamined biases to our decisions and that systems that rigorously examine the data are superior because they avoid these biases.  That general idea continues to have considerable support and I have no personal problem with the general idea.

I write, however, to raise several cautions that arise from the disastrous performance of the Super Crunchers in finance during the financial crisis.  Finance quants purported to be the best Super Crunchers in any commercial context.  We are examining therefore a series of hundreds of catastrophic failures at our most elite financial institutions by those who claimed to be the best Super Crunchers in existence.  (Physicists and the NSA’s top Super Crunchers always considered these claims to be laughable, but I will let them express that view.)

This article was prompted by reading a January 1, 2009 article entitled: “Irrational Expectations: How statistical thinking can lead us to better decisions” by “Deloitte University Press.”  Deloitte, of course, is “talking its book,” trying to sell its consultants as Super Crunchers, but the article is lively and worth reading.  The authors rightly consider the neoclassical model of decision-making a farce.

If you look at economics textbooks, you will learn that homo economicus can think like Albert Einstein, store as much memory as IBM’s Big Blue, and exercise the willpower of Mahatma Gandhi. Really. But the folks that we know are not like that. Real people have trouble with long division if they don’t have a calculator, sometimes forget their spouse’s birthday, and have a hangover on New Year’s Day. They are not homo economicus; they are homo sapiens.

Because they are, ultimately, quants incentivized to be sales guys, they oversell their quant products.

“But unlike a human decision-maker, a predictive model has the ability to optimally combine these and many other factors to efficiently estimate the employee’s relative likelihood of leaving. And unlike the human decision-maker, the predictive model will arrive at the same answer before and after lunch, takes virtually no time to draw conclusions, and is not affected by prejudices, pre-conceived ideas or cognitive biases. In short, predictive models can help us better approximate the ideally rational homo economicus.”

Here, two of the authors in Moral Markets could have helped protect the Deloitte authors from error.  The ringing phrase in Moral Markets is that “homo economicus is a sociopath.”  The Deloitte authors are businessmen who sell to other businessmen (and a very few women).  They apparently think that homo economicus is superior to a human – “ideally rational” and not “affected by prejudices” – or love, empathy, mercy, justice, or Tikkun Olam, but they are in fact describing a sociopath.

I explain that the financial crisis has demonstrated that financial CEOs and Super Crunchers have introduced the most characteristic financial traits of Wall Street and the City of London – greed and the desire for a “sure thing” – into our financial models.  It is true that the model will “arrive at the same answer before and after lunch” – the answer that enriches the CEO and the Super Torturers.  (This is a variant on the old joke that the accountant who gets hired is the one who responds to the question “what’s 2 + 2” with the answer “what would you like the answer to be?”)

The danger of hidden biases is, of course, exactly what believers in behavioral economics warn about – when it is in their financial interest to do so.  Deloitte’s authors illustrate that Super Crunchers are capable of writing odes to behavioral finance on p. 2 and ignoring those odes on the next page and pretending that “predictive models” are not designed by humans to have biases that will aid the humans.  Indeed, there is nothing as dangerous as someone who thinks (and tells senior managers) that their “Super Torturer” models are “not affected by prejudices, pre-conceived ideas or cognitive biases.”

But the Deloitte authors were still saying things at this point that were not in the heart of my work.  Then I hit this passage.

Many of Ayres’ examples are valuable in that they encourage one to think creatively about new ways in which predictive analytics can be applied. Everyone knows that credit scores outperform loan officers at assessing mortgage default risk.

The remainder of the paragraph goes on to describe predictive analytics models that the Deloitte authors consider relatively more esoteric, and therefore “creative.”  The second sentence is treated as a “duh” example – of course “everyone knows” (even your innumerate and nearly illiterate near do well cousin) that “credit scores outperform loan officers at assessing mortgage default risk.”  Remember that this sentence was published on January 1, 2009 – after the crisis had reached the acute phase.  The Deloitte Super Torturer disciples should have been inclined to caution in proclaiming that their financial models “approximate the ideally rational” rather than N. Gregory Mankiw’s famous illustration of “Mankiw morality” – “it would be irrational for operators of the savings and loans not to loot.”  (Another example of the “ideally rational” – the bank CEO as rational looter – made possible and amorally immoral by economic modelling of perfect rationality.)

The Deloitte authors, of course, knew that the financial “predictive analytics” had just produced results that purported to accurately value assets five digits to the right of the decimal point – but failed to come close to the actual values seven digits to the left of the decimal point that the models assigned to large pools of mortgage paper.  The models’ illusion of precision and objectivity also breeds complacency, creating a deadly mix that breeds recurrent financial crises.  The Fed’s longtime head of supervision, Richard Spillenkothen, gave an example of this problem in his memorandum to the Financial Crisis Inquiry Commission (FCIC).

Basel II was viewed by its most ardent Fed devotees with a quasi-theological reverence and as a sine qua non for assuring financial stability in an increasingly complex global financial system (p. 16).

Spillenkothen was talking about the Fed’s economists in that sentence, but he later makes the point that even skeptical Fed supervisors assumed that the credit rating agencies’ models couldn’t be systematically producing farcically inflated values.

Supervisors were not naïve enough to believe that external ratings were perfect, and they understood that downgrades were always possible – but going from triple A or super- senior to “junk” status or worthless overnight was simply never given serious consideration (p. 17).

The Deloitte authors have been well trained to see financial catastrophe as a marketing opportunity “in these tumultuous times.”

Consumer business:  We have helped companies use analytics to better understand their customers and sales patterns. While it is true that some companies make extensive use of their data to segment, target and cross-sell to their customers, we have found that many others use their data only to generate business metrics and fairly stale management reports. The situation is to a surprising degree similar to what we have found in the emerging field of workforce intelligence: the data exist but are not being used to refine decisions rooted in intuition and mental heuristics.  Analytics and predictive models can therefore be brought to bear to exploit the resulting market inefficiencies.

Mortgage triage:  We are assisting mortgage lenders to use predictive modeling to better identify potentially troubled loans before borrowers fall behind on their payments or default. In these tumultuous times, traditional reactive and subjective loan management methods are proving unsatisfactory. We are helping to bring predictive analytics to bear for mortgage lenders to design proactive loan and credit-line portfolio management strategies. Loans can be saved – and mortgagees can be kept in their homes – by strategically offering mitigation strategies before borrowers default.

(I leave the reality of how lenders and servicers created over 100,000 fraudulent affidavits to conduct foreclosures to the reader to consider.)

Ayres’ book gives an example of how bankers “exploit the resulting market inefficiencies” that arise when they can use data “to segment, target, and cross-sell to their customers.”

While most consumers now know that the sales price of a car can be negotiated, many do not know that auto lenders such as Ford Motor Credit or GMAC, often given dealers the option of marking up a borrower’s interest rate.  When a car buyer works with the dealer to arrange financing, the dealer normally sends the buyer’s credit information to a potential lender.  The lender then responds with a private message to the dealer that offers a “buy rate’ – the interest rate at which the lender is willing to lend.  Lenders will often pay a dealer – sometimes thousands of dollars – if the dealer can get the consumer to sign a loan at an inflated interest rate.  For example, Ford Motor Credit tells a dealer that it was willing to lend Susan money at a 6 percent interest rate, but they would pay the dealership $2,800 if the dealership could get Susan to sign an 11 percent loan.  The borrower would never be told ….(p. 143).

Ayres is famous for a study showing that auto dealers gave much worse deals to women and African-Americans.  I trust that at this juncture the reader understands that Deloitte’s gushing praise for this analytical technique as representing an “ideal” is overstated.  These techniques, absent effective regulation, will be used to detect “market inefficiencies” (aka, human weaknesses) and then to “optimally” “exploit” that inefficiency to enrich the bankers at the expense of the “inefficient” humans.  As Ayres helped show, the typical American “target[s]” that banks “exploit” are African-Americans, Latinos, and women.  But not to worry, the banks are using these techniques to seek to identify and “exploit” each of our weaknesses.  As Michael Corleone assured his brother in The Godfather, “It’s not personal, it’s strictly business.”  (The context was Michael’s plan to commit a double murder.)  Various organized criminals, while sociopaths, do have recognized restraints.  Those restraints may not be strictly moral, as Don Corleone explains in the movie he avoids the drug trade because joining it would make it harder to corrupt politicians.

Charles Keating, who professed to be a devout Catholic, deliberately targeted widows to exploit through the sale of worthless junk bonds of his insolvent holding company out of Lincoln Savings’ branches located near retirement communities.  Keating’s team figured out that that ultra-clean cut, well-dressed, and polite young men aged 18-20 were ideal to sell this toxic junk (which James Grant aptly labeled the worst security being sold in America) to the widows.  This was the first professional job for most of them so they had never been mentored to treat customers properly and they lacked both the experience and expertise to understand that what they were selling was worthless.  They were so callow that they sometimes enlisted their relatives to buy the worthless bonds.  But they looked great and sounded sincere when they repeated the sales pitches worked out by their seniors.  They looked like the grandsons the widows wished they had.  The worst banksters represent the sociopaths’ sociopath.

As for the glories of cross-selling, by the time Deloitte wrote its ode to maximizing cross-selling the practice was infamous in the UK banks.  UK banks cross-sold grotesquely improper products to their borrowers to an extent that is astonishing even if you have my background and interests in elite bank fraud.  Here are the primary takeaways that even regulators appointed by the Tories emphasize about the scam.  (The UK authorities religiously avoid using the “f” word to describe the bank abuses, even when they were clearly fraudulent.)

• The sales of PPI (a type of insurance) to individuals and swaps to small business owners (you know, the entrepreneurs bankers always claim to be helping) were the overwhelming source of UK banks’ total profits
• The sales of PPI and swaps were overwhelmingly inappropriate (claimants are winning 80-90% of their claims in front of the Ombudsman
• The markup on PPI was an extraordinary 80%

In sum, the paramount business strategy of the massive UK bank (and many smaller ones) was to systematically and repeatedly rip off their customers.  The banks’ compensation systems and informal and formal forms of discipline forced employees to rip off their customers on a daily basis.  One strategy was to force an employee whose ethical restraints caused her to “fail” to cross-sell to keep a cabbage (signifying in the UK a dumb, useless person) on her desk until she joined in ripping-off customers.  The purpose was to humiliate the employee and force her either to act unethically or resign.

Models Can Be Crafted to Aid Fraud

And then there is the, vastly larger, home lending analog to Ayres’ example of how auto dealers and lenders corruptly conspire to rip off their customers.  Ayres, of course, is not supporting the auto dealers and lenders’ conspiracy to rip off the customers.  But he is not immune to the lies they tell.  Consider how Ayres phrased the matter:  “When a car buyer works with the dealer to arrange financing,” That is the exact phrase that the dealer uses to deceive his or her customer – “I’ll work with you.”  As Ayres explains, however, this is a cynical lie.  The dealer is working against, rather than “with” the customer.  The lender bribes the dealer to deceive and “exploit” the customer.

Precisely this same scam was used by fraudulent lenders to corrupt loan brokers by creating intense, perverse incentives for the broker to do five things.  The bribe from the lender to the broker was called the “yield spread premium” (YSP).  The lenders’ controlling officers intended the bribe to create five perverse incentives among mortgage brokers.

1. Hustle relentlessly to find induce people, to buy a home and arrange a loan through the broker regardless of whether the borrower could afford to repay the loan.
2. Induce them to take out a liar’s loan, so that the borrower would have to pay a higher interest rate
3. Induce the broker to inflate the borrower’s income to make the loan look prudent
4. Induce the broker to extort the appraiser to inflate the appraised value of the home in order to make the loan look prudent
5. Induce the broker to charge the borrower a materially higher interest rate.

In fairness to real estate brokers, the fact that they have been bribed was actually typically disclosed due to federal mortgage disclosure requirements.  However, only someone sophisticated in real estate would understand that the disclosure indicated that the customer had been ripped off.  Most borrowers relied on their broker to direct them on how to fill out the forms and what the RESPA disclosures meant.  Brokers had powerful incentives not to disclose that “this YSP figure here means the lender bribed me and my brokerage firm to induce you to pay an excessive interest rate.”

Krystofiak Tries to Save America (and the Fed) and Shows How the Models Torture Facts

All of this would have been clear to the Fed (which had unique authority under HOEPA – since 1994 – to ban all liar’s loans), Ayres, the Deloitte authors, and Attorney General Eric Holder had they ever read Steven Krystofiak’s 2006 testimony to the Fed about liar’s loans.  Here is how his written statement began:

“My name is Steven Krystofiak, President of the Mortgage Brokers Association for Responsible Lending. The MBARL is an advocacy group protecting consumers and the loan industry from outlandish and counter-productive loan programs. Currently we see stated income and stated asset loans as the largest problem in the real estate industry.”

Krystofiak was trying to save the Nation.  He went to the regional Fed hearing (mandated by Congress) as an uninvited participant.  He was given a few minutes to speak and asked no questions by the Fed.  All the quotes here are from his written statement to the Fed.  The Mortgage Bankers Association (MBA) chose as its spokesperson at the hearing (who was invited – and praised lavishly by the clueless Fed Governor) – a senior officer from one of the largest specialty lenders of fraudulent “liar’s” loans.  Note that the MBA chose this S&L, IndyMac, to provide its official spokesperson on the issue of liar’s loans to the Fed.  In 2006, reporting on Krystofiak’s study, the MBA’s own anti-fraud experts (MARI) warned every member of the MBA in writing that the loans were 90% fraudulent.

Krystofiak’s got right to the point.  His next statement was:

“A stated income loan is a loan where the income that is put on a home loan application is not verified at all by the banks. The banks simply take your word for it. Home buyers might be unaware of the fraudulent income that is being stated on the loan application because the loan officer, or bank representative have the power to falsify the income on the application.”

While Krysofiak did not use the famous industry phrase that “a rolling loan gathers no loss” he explained the concept to the Fed.

“This cycle is the reason why currently there is a small default rate on stated income loans. Once appreciation goes flat the cycle of “cashing out” [mortgages through refinancing] will no longer keep default rates low. Stated income loans need to stop now before thousands of new home buyers buy property that they cannot afford.”

Krystofiak then demonstrated that the banks were “accounting control frauds.”

“3. Fraud is encouraged by the banks

A large problem as to why these loans have become so prevalent is because the first line of defense against stated income loan fraud are individuals who are commission based; the loan originator, the bank representative, and in many cases the managers for the bank reps have a large portion of their income derived from bonuses based on loan production. Bank employees, i.e. underwriters and bank processors, return applications back to mortgage brokers with instructions to send back an application with a higher stated income. The mortgage industry has become comfortable with stating incomes higher on loan applications.”

It is, of course, the bank’s and loan brokerage’s controlling officers who shaped there perverse incentives and continued them even as they received myriad reports of endemic fraud.

Krystofiak then explained the reality of the “ideal” model that Deloitte claimed “arrive at the same answer before and after lunch, takes virtually no time to draw conclusions, and is not affected by prejudices, pre-conceived ideas or cognitive biases ….”  Well, that’s true if (a) no one devises the model to game it, (b) no one changes the model to game it, and (c) no one changes the inputs.  Krystofiak implicitly addressed the first two points by explaining that the values were Super Tortured by the lenders’ agents to inflate values and the losses were hidden by refinancing.  He now explained how inputs were gamed – and that the models were designed to allow them to be repeatedly gamed.

“Online underwriting systems that are used by Fannie Mae and some banks are being exploited by bank representatives and loan officers wanting to obtain a loan with stated income underwriting standards but with fully documented interest rates. The systems allow mortgage brokers to “play” with different incomes more than 15 times until they get the results they want.”

The reality is that the answer varied up to 15 times – because the model was designed to be gamed.

Krystofiak realized something that economists did not recognize (or pretended not to) about the interconnection between the rise in liar’s loans, the hyper-inflation of the real estate bubble, and the delayed recognition of massive losses on the pervasively fraudulent liar’s loans.  One must imagine the Fed economists choking as a young mortgage broker dared to explain real estate economics and reality to them.  He also introduces the role of more sophisticated real estate speculators.

“6. Stated income loans are why home prices have skyrocketed. They have caused a large demand in the US housing supply.

Many economists are currently unaware of how prevalent stated income loans are. They attribute high home prices to low interest rates, low to zero percent down payments, speculative purchasing, and interest only loans. What economist[s] fail to realize is that popular stated income loans are what have led have home prices to skyrocket in recent years. Traditionally banks told consumers how much money their maximum monthly payment could be, based on their income and outstanding debts. With stated income loans the consumer is in the driver’s seat to tell the bank how much they want to spend every month on their home payments. This is dangerous and unproven over time and is the reason [housing prices] have doubled while the median incomes in the respective areas have remained relatively flat since 2001. Loose underwriting guidelines caused by stated income loans have allowed individuals to speculate on 3, 4, and in some cases over 5 homes at once.

Homes have a unique situation where demand is directly related to whether or not someone can receive a loan from a bank or lender. With banks loosening their guidelines for the home buying process we have experienced a huge surge in demand for homes over the past few years.”

Krystofiak added the fact that appraisal fraud was endemic – and created overwhelmingly by lenders and reiterated the FBI’s warnings about mortgage fraud.  He then made two points – the Fed needed to stop liar’s loans now – and if they failed to do so it would blow up in their faces.  To do so, he resurrected the lead data he had buried.

“13. Stated income loans must stop now

Stated income loans might be more convenient for a small portion of the home buying population, but it is a sleeping plague on the financial integrity for the rest of us. Federal regulators must end stated income loans now. Greed by mortgage brokers, banks, and real estate developers must not be encouraged by keeping this issue unaddressed and silent. Stated income loans are being used fraudulently in alarmingly high rates and are hurting consumers. If federal regulators don’t act on this now, they will be dealing with the consequences of their lack of actions later. 

Data Collected by the Mortgage Brokers Association for Responsible Lending

58. A recent sample of 100 stated income loans which were compared to IRS records (which is allowed through IRS forms 4506, but hardly done) found that 90% of the income was exaggerated by 5% or more. MORE DISTURBINGLY, ALMOST 60% OF THE STATED AMOUNTS WERE EXAGGERATED BY MORE THAN 50%. These results suggest that the stated income loans deserves the nickname used by many in the industry, the ‘liar’s loan’ (emphasis in original). “

Note that the Fed could have ordered the lenders it regulated (including those it regulates as part of holding companies) to conduct the Krystofiak test at their institutions because nearly all borrowers signed the 4506-T form to get their loans.  For obvious reasons, people rarely deliberately inflate the income they report to the IRS.  Form 4506-T allows the bank to get a “transcript” of the borrower’s tax return, allowing an easy, inexpensive, and highly accurate means of preventing the inflation of the borrower’s income in a loan application.  Lenders making liar’s loans, however, virtually never used their ability pursuant to the 4506-T to obtain the transcript of the borrower’s tax return and verify the borrower’s income.  Doing so would have inhibited their fraud schemes.

The Fed and Folly of the Models

The Fed’s ignoring of Krystofiak’s warnings was nonsensical but consistent.  If the Fed’s senior leadership were to admit that liar’s loans were pervasively fraudulent it would have no choice but to recognize an imminent economic catastrophe.  Here is what the Fed leadership knew (no later than 2005) about the extraordinary (and growing) prevalence of liar’s loans at the largest banks.

Sabeth Siddique, the assistant director for credit risk in the Division of Banking Supervision and Regulation at the Federal Reserve Board, was charged with investigating how broadly loan patterns were changing. He took the questions directly to large banks in 2005 and asked them how many of which kinds of loans they were making.  Siddique found the information he received “very alarming,” he told the Commission. In fact, nontraditional loans made up 59 percent of originations at Countrywide, 58 percent at Wells Fargo, 51 at National City, 31% at Washington Mutual, 26.5% at CitiFinancial, and 28.3% at Bank of America. Moreover, the banks expected that their originations of nontraditional loans would rise by 17% in 2005 to 608.5 billion. The review also noted the “slowly deteriorating quality of loans due to loosening underwriting standards.” In addition, it found that two-thirds of the nontraditional loans made by the banks in 2003 had been of the stated-income, minimal documentation variety known as liar loans, which had a particularly great likelihood of going sour.

The reaction to Siddique’s briefing was mixed. Federal Reserve Governor Bies recalled the response by the Fed governors and regional board directors as divided from the beginning. “Some people on the board and regional presidents . . . just wanted to come to a different answer. So they did ignore it, or the full thrust of it,” she told the Commission.

Within the Fed, the debate grew heated and emotional, Siddique recalled.  “It got very personal,” he told the Commission.  The ideological turf war lasted more than a year, while the number of nontraditional loans kept growing…. (FCIC 2011: 20-21).

Yes, that was the Fed under Alan Greenspan and Ben Bernanke.  They had all kinds of models telling them everything was wonderful.  The models produce faux data that is massively inflated.  Then they had actual data, interpreted by experienced examiners and supervisors that caused them to warn that a disaster was coming.  Guess which source Greenspan and Bernanke chose to rely on and which it chose to assault the messenger?

In my most recent article I cited portions of Richard Spillenkothen’s written memorandum to the FCIC about the Fed’s economists’ insane efforts to virtually eliminate capital requirements for the largest banks through their Basel II (Super Cruncher) models.  I was responding to a Cato author’s claim that Basel II demonstrates the insanity of regulation.  I made the point that the problem with the Fed was that its key economics decision-makers shared Cato’s key anti-regulatory dogmas.  I did not quote this portion of Spillenkothen’s explanation that discusses both the Fed economists’ dogma and their faith-based models.

“Over the last 15 years, professional economists have played an increasingly important role in the Fed’s supervision function – with respect to both policy formation and program execution – and it seems clear that this role will expand further in the years ahead.

[Fed] economists’ overarching intellectual commitment to the ideal of efficient and self-correcting markets, abiding faith in counterparty and market discipline, inherent skepticism of supervision and regulation, and penchant for solutions based on complex modeling or arcane quantitative risk measurement methodologies were significant contributing factors to some of the major regulatory policy errors since the mid 1990s.

Going forward, it will be important to structure and manage supervision in a way that garners the important benefits of economists’ perspectives, techniques, and expertise – without diluting the quality of hands-on, micro-prudential financial oversight; without basing policies on philosophical aspirations or theoretical constructs at odds with the realities of risk management or the lessons of history; and without diverting supervisory resources from critical safety and soundness priorities” (p. 29).

Bank and S&L examiners have repeatedly proved their ability to outperform the models – to identify the frauds while the models’ results are still praising the frauds.  George Akerlof and Paul Romer emphasized this point in their famous 1993 article on “looting.”

“The S&L crisis, however, was also caused by misunderstanding. Neither the public nor economists foresaw that the regulations of the 1980s were bound to produce looting. Nor, unaware of the concept, could they have known how serious it would be. Thus the regulators in the field who understood what was happening from the beginning found lukewarm support, at best, for their cause. Now we know better. If we learn from experience, history need not repeat itself” (Akerlof & Romer 1993: 60).

Spillenkothen makes a related point.

“[S]upervisors, economists, and researchers have spent decades trying to develop improved surveillance tools, early warning systems, and better predictors of future bank performance. During my career, the evidence suggested that surveillance tools, even those that incorporated market information, were useful in reflecting a bank’s current condition; but were not necessarily good predictors of future performance. And experience also suggested that examination findings and supervisory judgments consistently identified problems before they were reflected in market indices, which tended to be lagging indicators.

Earlier this decade, in connection with an effort to explore ways to make greater use of public information in the supervision process, an attempt was made to find examples of where the market identified problems before they were noted in supervisory examinations. To the best of my recollection, no examples were found” (p. 22).

Remember, this study included Fed economists devoted to the dogma that regulators are useless and private market discipline is elegant, reliable, and robust – yet they could find no examples of their dogma having any basis in reality.  The Fed economists, of course, did not abandon their dogmas in response to reality.

Human Underwriters Were Mighty Good When They Had Honest Leaders

Deloitte’s assertion, without benefit of argument or citation, that: “Everyone knows that credit scores outperform loan officers at assessing mortgage default risk” is not true because I don’t know any such thing.  I will simply note in passing that two significant problems demonstrated during the crisis were that credit scores can be gamed and falsely attributed to the wrong person.  Empirical studies confirm a suspicious pattern with undue numbers of credit scores just above the cutoffs used by many lenders.

The point I emphasize here is that while a credit score is useful and underwriters’ review them, a credit score inherently cannot evaluate one of the “C’s” essential to successful underwriting – the borrower’s “capacity” to repay the loan.  This was precisely the problem with liar’s loans, because they did not verify the borrower’s income and were pervasively fraudulent as Krystofiak warned the Fed and the Nation.  Further, the fact that “a rolling loan gathers no loss” means as Krystofiak explained that the tests that purport to prove that relying on the credit score provides a superior default prediction to also evaluating “capacity” are unreliable for the reasons Krystofiak explained.  This is why models proved so embarrassing in massively understating the losses inherent from liar’s loans.  It also explains why our (Office of Thrift Supervision – West Region) examiners’ conclusions in 1990 that liar’s loans should be banned that led us to begin driving out of the S&L industry in 1991 were so superior to the banksters’ claims that they could rely on high credit scores to “compensate” for failing to verify the borrower’s actual income.  What we actually know is that relying on credit scores to make liar’s loans is vastly worse than relying on honest, competent underwriters.

For example, Countrywide had a Super Cruncher model (that did not rely fully on credit scores).  One could argue that it performed in a way that was superior to Countrywide’s loan officers for the model called for the rejection of hundreds of thousands of loans (most of them terrible) that were actually approved by humans as “exceptions.”  But the comparison is not reliable.  The model was a Potemkin model designed (at least from the perspective of Countrywide’s controlling managers) to fool the regulators and other outsiders.  Exceptions were endemic because Countrywide was following the accounting control fraud recipe and it is essential that the leaders gut the underwriting function and suborn the controls.  The comparison of model v. human becomes nonsensical in such situations because the models are (to stay with my Russian theme) a Maskirovka designed to hide the truth.  The humans running the fraud will insure that the models fail or are ignored.

How good were human underwriters when they did not suffer from the perverse incentives created by their fraudulent bosses?  Exceptionally good is the answer.  Vastly better than the performance of the models about the liar’s loans – even if those models had not been ignored when they called for loan rejection.  I note for purposes of potential bias that I was once an active member of the Federal Home Loan Bank of San Francisco’s (FHLBSF) credit committee.  Before Fannie was privatized, and before Freddie was created, Fannie created a national system of high quality underwriting.

A study by the FHLB Chicago examined charge-offs on mortgage loans made by FHLB Chicago thrifts and Fannie and Freddie in this earlier era.  Figures 2 (Fannie and Freddie: 1975-1998) & 3 (the FHLBs: 1994-1998) on p. 33 are the keys.  The average annual loss at Fannie and Freddie for this 23 year period was 5 basis points (0.05%).  The most extreme loss for the entire 24-year period was 16 basis points.

The FHLB Chicago thrifts’ average annual loss from (1994-1998) was 3 basis points.  The FHLB thrifts were relatively small and were particularly unlikely to have fancy models.  Humans can be taught, particularly in a well-run bank with effective internal controls and honest managers, to make conventional home loans that will rarely default.

Fraud Epidemics and Models that Rely on Distributions

What we should have learned from the crisis is a number of cautions about financial models.  I’ll leave the non-ergodic nature of the data feeding the models to the quants and concentrate on an understandable, critical point.  There is no fixed distribution of bad events in business.  One of the most important reasons this is true is control fraud epidemics.  This is the importance of the concept of a “criminogenic environment.”  If the incentives are sufficiently perverse, such frauds can become epidemic, particularly if they create “Gresham’s” dynamics in which bad ethics drives good ethics out of the markets and professions.

Here is the Free Dictionary’s example of a symmetrical distribution.  It illustrates a situation in which most of the distribution (say of the height of 15 year-old females) is clustered fairly close to the average (mean) height.  To either extreme side are the “thin” “tails” of the distribution – a small percentage of females at age 15 are above 6’5” or below 4’8.”  In finance, it is usually the tail that kills your company.  This would be the very unlikely event that caused extreme losses.

A static distribution is not obviously insane for a characteristic like height, though if one examines the substantial increases in average height in regions within countries in Asia that have been most effective in reducing childhood malnutrition it would be clear that the distribution of height is dynamic rather than fixed (static).  Many forms of crime, including accounting control fraud are extremely dynamic.  We understand this in the street crime context.  If we leave a nice car in a “bad neighborhood” unlocked with the key in the ignition and park it far from any street lights the chances of it being stolen go up enormously.  If you go out drinking in seedy bar in a “bad neighborhood,” brag about how much money you have, flash a fat bankroll, get drunk, stagger out the door and walk into a dark alley the chances of getting robbed go up dramatically.

If accounting control fraud becomes epidemic the likelihood of a bubble hyper-inflating (absent vigorous regulatory intervention) goes up greatly.  Because “a rolling loan gathers no loss” the loss recognition will be greatly delayed as long as the bubble expands.  Models that rely on recent performance will seriously understate risk (and in finance that means overstate market values).  The three “sure things” of accounting control fraud include “severe losses.”  All financial bubbles end and when they end the fraudulent loans can no longer be hidden by refinancing.  Extreme losses in these circumstances are not improbable, but virtually certain.  (In jargon, there is no true exogenous distribution of accounting control fraud.  Our public policies determine the likelihood of an epidemic of accounting control fraud developing and hyper-inflating a bubble.

Conclusion

The finance models tend to be lagging indicators of crises.  Indeed, they are so bad and so biased that they contribute to causing the crises.  As I have explained in prior articles, standard econometric studies are criminogenic if there is an epidemic of accounting control fraud because the studies rely on reported profits or stock prices (which are heavily affected by reported profits).  Economists consistently recommended policies on the basis of such econometric studies that produced disastrous results.  In a regression analysis, whatever policies most aid accounting control fraud will show the most powerful correlation with higher reported profits.

Examiners have a far better track record of being the first to warn of developing systemic risks.  Naturally, our response to the crisis was to put economists, and their models, in charge of identifying systemic financial risk.  Economists and their models are expert at creating severe systemic risk and impeding regulators from stopping such risks.

Bernanke put one of the Fed’s most anti-regulatory, failed economists in charge of Fed supervision to ensure that Fed supervision would be further degraded.  In a very funny line for an agency utterly dominated by anti-regulatory economists in which Spillenkothen was the only seriously different voice that Greenspan and Bernanke heard, Bernanke claimed that appointing an economist with no supervisory experience the head of Fed supervision was a great leap forward because it would make supervision multi-disciplinary.  Instead, as Bernanke intended, appointing an anti-regulatory economist as the Fed’s top supervisor removed the only major source of multi-disciplinary thought among the Fed’s senior staff.