Can Open Source Ratings Break the Ratings Agency Oligopoly?

Yves here. One of the causes of the financial that should have been relatively easy to fix was the over-reliance on ratings agencies. They wield considerable power, suffer from poor incentives, in particular, that they can do terrible work yet are at no risk of being fired thanks to their oligopoly position, and are seldom exposed to liability (they have bizarrely been able to argue that their research is journalistic opinion, which gives them a First Amendment exemption). But they are not big enough moneybags to be influential donors, nor are they critical to the financial infrastructure.

Yet they’ve managed to stymie meaningful reforms. Scarecrow and Jane Hamsher detailed how Standard & Poors started threatening to downgrade US debt just as provisions in Dodd Frank that would have made them liable for their opinions, just like other experts, were moving towards a vote. And, mirabile dictu, they managed to get that provision stripped from the bill.

Note that the EU is in the process of imposing rules that would make the ratings agencies liable for “mistakes in case of negligence or intent.” This presumably would apply only on ratings of issuers based in the Eurozone; if a European investor relied on ratings to invest in a US security, these rules would not apply. Some commentators are skeptical of other provisions, namely, ones to curb sovereign debt ratings and restrict ownership of ratings agencies.

At the same time, even though ratings of structured products have proven to be sorely wanting, investors still prefer having a bad metric to no metric, particularly since that allows them to shift blame if Something Bad Happens (“everyone else in the industry uses them, we would have lost business if we tried something different”).

One way to remedy this problem would be to reduce the power of the current incumbents by allowing for new entrants. While the SEC initially appeared to be receptive to that approach, people I know with solid backgrounds have found that the SEC is requiring so much capital as to make it unattractive to proceed, thus keeping the current oligopoly intact.

Open source ratings is another route for chipping away at the oligopoly. In computers, privately developed operating systems also had a quasi monopoly position, yet open source programs have come to play a more and more important role. And many now forget that in the early days of Linux, most potential users regarded it as simply nuts to consider it. How could you use software where no one was….responsible??? Where there was no business that had an interest in updates and bug fixes? Who do you call if something goes wrong? (That ignores the issue that Microsoft in particular releases buggy software, and that many fine programs nevertheless don’t get updates and upgrades because the developer can’t sell enough copies to make a business out of it). But over time, Linux and other open source software has gone from being bleeding edge to having meaningful market share in certain product areas. So if more independent-minded investors were to find open-source ratings useful (hedge funds, sophisticated individuals and family offices), you could see acceptance there leading it to become a legitimate alternative for institutional investors.

By Professor Krassimir Petrov, who has taught economics and finance in the U.S., Ireland, Bulgaria, Saudi Arabia, Bahrain, Taiwan, and Macau

The current credit-ratings system is a complete farce that has caused damage in the trillions. It needs to be completely reformed into a more transparent, competitive system. An open-source approach presents a perfect solution.

Introduction

Credit ratings are of paramount importance in the economy. A rating evaluates the creditworthiness of the borrower. It provides the likelihood that the borrower will repay its debt, which describes the risk of default, and therefore the risk of the borrower and the loan. This risk determines the required return on the loan – the interest rate. Because investors cannot adequately determine the credit rating of a borrower, they rely on the ratings provided by credit ratings agencies. Good ratings allow for proper allocation of capital; while poor ratings result in overlending, misallocation of capital, and investor losses. Thus, proper credit ratings make capital markets efficient and underpin strong economic growth.

Sovereign credit ratings are especially important, because they affect the whole credit system in the economy. The sovereign is generally the government of an independent country like the United States, France, or Argentina. In general, the government is considered to be the lowest risk in the economy; in a sense, it represents the relative risk-free rate in the economy. Corporations and municipalities derive their risk from their national government. They are generally considered higher risk, because they “inherit” their government’s risk and add on top of that other economic and business risk. The whole credit structure of an economy is entirely built upon government credit. Its risk is based upon government risk. Any misjudgment of government risk propagates throughout the whole credit system with catastrophic results for the economy and horrific losses for the lenders.

Unfortunately, our modern credit rating system is fraught with problems that have already resulted in trillion-dollar losses and untold suffering by many nations. These problems extend to municipal, corporate, and structured-finance ratings. The system has proven to be a massive failure and needs to be entirely overhauled. The reform must necessarily provide for more transparency, more competition, and more flexibility. It also needs to address a large number of other problems outlined in the next section.

Problems of the Credit Rating System

The weaknesses of the current credit rating system are well-known and well-understood. It is not surprising that the ratings agencies have been heavily criticized for their spectacular failures in recent years. Triple-A ratings were assigned to large volumes of “toxic” mortgage-backed securities and collateralized debt obligations that ultimately lost value. A classic example of a corporate rating failure is that of Enron, where the ratings agencies did not downgrade the company until a few days before its bankruptcy. Equally spectacular were the triple-A monoline insurers like MBIA and AMBAC that went bankrupt or suffered belated multiple-notch downgrades during the global financial crisis. Finally, the spectacle surrounding the downgrade of the U.S. Government during the debt-ceiling crisis in 2011 made it all too clear that credit ratings agencies are politicized and incapable of prompt, decisive action. A deeper investigation offers other, more subtle problems. Here is a brief outline.

Problem 1 – Government Monopoly. The credit ratings agencies enjoy an extraordinary government monopoly that shields them from effective competition. With the creation of federal deposit insurance and the rise of mutual funds, the government began to regulate heavily the financial industry. It was necessary to limit risky exposures, including “risky” bonds. So, with the logical questions of “what is risky” and “who would determine it”, the government provided the exclusive privilege to a handful of “Nationally Recognized Statistical Rating Organizations” (NRSROs), thus shielding these corporations from competition. It created a de-facto cartel with long-lasting ramifications.

Problem 2 – Unaccountability. When ratings agencies make mistakes, they are not accountable for them. They simply don’t pay for their errors. They are not subject to free market forces either. They sell credit ratings and they charge for their product no matter how bad it is. In the free market, when the product is bad, the customer refuses to buy and switches to another provider. However, the exclusive privilege provided by the government excludes any competition; the consumer has no choice, no matter how bad the product is.

Problem 3 – Politicization. The rating process has become completely politicized. Because the government provides the “protection” (the privilege), the government can also take it away whenever the agency does not serve its interest. In essence, the master of the credit ratings agency is not the customer, but the government. This means that the government has extraordinary power and influence over the agencies, but government itself is driven by politics.

Problem 4 – Conflicts of Interest. The fact that the issuer pays for the rating provides a strong bias for a higher rating. Because the borrower pays, not the lender, the built-in incentive is to serve the interest of the borrower and to assign a better/higher credit rating. This clearly provides a conflict of interest, as the interest of the lender is not really protected and the lender has no recourse for a bad rating.

Problem 5 – Lack of Transparency. This really helps the agencies to promote their agenda as they see fit. They basically decide when to change the rating, how, and under what circumstances. Sometimes they “warn”, gauging the political winds and the potential for backlash. When needed, political favors or concessions are made to attract or retain large corporate clients.

Problem 6 – Inconsistent Methodologies. In scientific research and development, verification of methodology and replication of results is a required condition for proper scientific inquiry. In other words, consistent methodologies produce consistent results. This neatly ties in with the problem of transparency. Even though rating agencies have published their methodologies, these procedures lack rigorous validation and peer review. Also, agencies do not publish the minutes of their rating committees. One common source of inconsistency is the intentional use of too many factors, which allows raters to emphasize some credit drivers, while conveniently ignoring others. Thus, the agencies do not consistently follow their own methodologies. Naturally, inconsistent methodologies would lead to inconsistent ratings.

Problem 7 – Inconsistent Ratings. While major factors determining the rating of an entity are well-known and well-understood, such as the overall debt of a government, ZeroHedge has pinpointed a troubling inconsistency of BBB-rated countries that illustrates the problem in an article aptly-named “BBBeauty Is certainly In The Eye Of The BBBeholder”. The inconsistency is quite glaring, with Kazakhstan’s debt level of only 11% and Ireland’s – of 106% for similar credit ratings. Of course, government debt is not the only factor determining the rating, but clearly it is the dominant factor. Indeed, a variation in the factor is natural, but such a wide variation does not make common sense. It appears that these variations are more driven by politics and geopolitical concerns than by sound financial analysis.

Problem 8 – Untimely Downgrades. Downgrades are quite often late; and sometimes they come disastrously late, as many investors found out with the ratings of the credit derivative instruments. Most often, the agencies are very unwilling to downgrade, so that they don’t hurt their corporate clients. With sovereign ratings, political and geopolitical considerations delay the fateful decision to downgrade. Sometimes the delay comes from a fear of rocking the markets… And while the agencies habitually delay the downgrade, they sometimes delay the upgrade of many state and local governments. For example, as a result of the increased municipal bond defaults during the Great Depression, the agencies were too slow to incorporate improved credit performance of municipal debt after WWII – even half a century later.

Problem 9 – Insufficient Analysis. Sometimes the analysis is insufficient. Sometimes the responsible employees are overworked, too burdened with responsibilities, too slow to respond, and lacking sufficient time and energy. Sometimes the employee is young and inexperienced. And sometimes the outcome is politically pre-determined, so there is no need for proper analysis.

Problem 10 – Ignoring Maturity. Sovereign credit ratings should follow the example of corporate debt – each bond issue should have its own rating. Each rating should be based on maturity. To illustrate this important point, we can consider France, whose debt/GDP ratio is rapidly approaching 100%. While no one currently suggests or expects France to default within 1-3 years, many analysts believe that there is a much higher likelihood of default in the 10-15 year time frame. Thus, as an illustration, without suggesting that these rating are currently appropriate for France, maturity 0-5 years is rated “AAA”, maturity 5-10 is rated “AA”, and maturity 10-20 is rated “A”.

The Open Source Solution to Credit Ratings

The open-source movement has proven to be a powerful force in our modern technological world. It began in the software industry and rose to prominence with the success of Linux. It is steadily spreading in other areas. Open-Source Credit Rating Models provide a superior platform that effectively resolves all of the above problems. One leading example in the field is Public Sector Credit Solutions, which published in May 2012 an open-source sovereign and municipal rating tool called the Public Sector Credit Framework.

It is important to understand that, just like Linux, the open-source model does not immediately provide the “ultimate” solution. However, it is a dynamic, evolving rating system that is open to enhancements and modifications. Anyone can modify it, justify their enhancements, and observe the results. It is likely to branch, just like Linux, into many alternative and competitive versions. Some versions will prove better and will stand the test of time; others will fall behind and will be forgotten; still others might specialize in municipal, corporate, or structured credit. In such an open, transparent, and competitive environment, natural selection, adaptation, specialization, and survival of the fittest will determine the winners.

The open-source solution addresses adequately each one of the above concerns:

1. Open Competition. The open-source model breaks the status-quo cartel of the big three agencies. Anyone can enter the business and anyone can modify it. True, the monopoly is based entirely on government privilege, so open-source cannot break it by sheer legal force, but can still provide an option that the market can adopt, develop, and follow on its own.

2. Accountability. Competition forces accountability. The model has to work; it has to be good. If it is not, then a competitor will make a better product and promote its superiority. Open competition forces everyone to innovate. It keeps the competition honest – your competitor can always point to your flaws. It also provides the customer with the option to switch to a more accountable competitor. If competition forces accountability, then the customer enforces it!

3. Non-Political. Open-source is apolitical. Everyone can see what it is, how it works, and its final outcome. Transparency keeps the system out of politics. Whatever the politics might be, if the model is transparent, it delivers automatically its rating as a quantitative judgment, as a probability of default, just like SAT provides a score (number) to the student.

4. No Conflict of Interest. Open-source resolves immediately the problem of conflict of interest. The agency would follow an open methodology. Transparency will keep agencies honest. The customer/investor could run the model on his own and confirm the rating. The less conflicted competitor can point out the conflict. A whole new industry can grow to serve the needs of the investors, not of the issuers. That’s how it was many decades ago, and that’s how it should be. Open-source can facilitate and accelerate this process.

5. Transparency. By far, transparency is the biggest advantage of the open-source solution. It is the foundation that resolves other major problems of the current system. Transparency forces objectivity – everyone can see what the rating is and how we get it. It allows participants to pinpoint specific flaws in the system and improve on them. Transparency makes the rating automatic, thus keeping it out of politics. Automatic ratings also solves the problem with timeliness.

6. Consistent Methodology. Transparency exposes inconsistencies and enforces consistency. This does not mean that the methodology is static and does not evolve. It means that the same criteria for Mexico are applied to the U.S., just like the same criteria for Ireland are applied for Kazakhstan. As the methodology changes or improves, it is applied equally to everyone.

7. Consistent Ratings. Transparency ensures consistent methodology, which in turn ensures consistent ratings. If we use the same formula and the same inputs, we always get the same result. It is like McDonald’s using the same Big Mac recipe around the world to get the same burger. If the inputs and methodologies are consistent, then so is the rating.

8. Timely Downgrades. Transparency, consistency, and the use of data-driven software tools make the ratings automatic. When new economic data is published, the model can be re-run with immediate results. No more delay and procrastination. This also keeps politics out of the equation.

9. Sufficient Analysis. The open-source competition ensures that the model is optimal and that the analysis is adequate. If it is not adequate, then the model will be improved upon by competitors who will seize on the opportunity.

10. Incorporating Maturity. The Open Source Credit Model (developed by the PSCS) calculates a probability of default for all maturities and automatically assigns a rating for each maturity. Of course, that’s how every probability model works – probability can be calculated only for a specific period of time; without a time frame, probability is meaningless. Thus, the model automatically assigns ratings by maturity.

Conclusion

The current credit rating system is fraught with problems: from lack of competition, accountability and transparency, to politicization, inconsistencies, and untimeliness. Private sector competition and transparency are needed to overcome these problems. The open-source approach, as implemented by the Public Sector Credit Framework, provides one possible solution to our current credit rating problems. Other solutions could be possible, if they offer sufficient degree of competition and transparency.

Print Friendly, PDF & Email

16 comments

  1. Conscience of a Conservative

    That the rating agencies are deemed to provide any value in the judgement of risk would seem to be based on the following premises, (they have smarter people, faster computers , or access to inside information), none of these would apear to be the case, although maybe there’s a weak argument for the last point. Since this is not the case the need for nationally recognized rating agencies is poor at best. Personally I don’t see how they can be taken seriously in the absence of them performing an auditing task on the very data itself(i.e. sampling the accuracy of loan data in an ABS or an accounting number on a muni), and when the ratings come under fire, they claim they are not auditors but more akin ot a news paper publishing an opinion (which was paid for). They would be better off auditing the data, and publishing models without providing a graded result, that is nothing more than a letter built upon assumption, built upon another assumption which is meaningless when the world does not travel down a single path.

    Rating agencies warning of risk and downgraded debt is basically like showing up to a battlefield after a fight and shooting the wounded.

  2. LeeAnne

    The current credit-ratings system is a complete farce that has caused damage in the trillions.

    … trillions of people? that would be about right if this isn’t clawed back and the system that created it demolished with penalties.

  3. John F. Opie

    First, a disclaimer: I work for a rating agency in Europe, but not one of those criticized here. Our work is forecast-oriented and we provide transparency to our customers, but we’d be crazy to post our algorithms, since all this would do would be to create competitors that would crunch numbers without understanding what they are doing.

    While I heartily agree that the anglo-saxon rating agencies have screwed up monumentally, the idea of open-sourcing this won’t work.

    The reason? The work involved is non-trivial. Seriously non-trivial, and making changes to methodologies can have far-reaching effects, ones that take a significant skill set to understand properly.

    You can make computer code open source and if it doesn’t work, or, more exactly, if it gives you strange answers, no big deal unless you are applying it to situations where errors can cause major health or financial disaster, since there is no one who is, at the end of the day, responsible. Kill someone because the code caused a machine to work improperly, or destroy a company because the code led to a rating move from investment to junk grade, but have all be the result of folks failing to understand that they were using faulty code, worst possible outcome and everyone involved can point the open source and blame that.

    Nope. We’ve seen that what damage can be done by investment decisions made based on really piss-poor ratings. Some, if not most, of the advantages listed here reflect not what happens if you use open-source methodologies, but rather what you get if you are honest in what you do and let your customers see how you reach your conclusions.

    The other real problem with open-source here is that the moment you make significant changes to the rating methodologies, they become non-comparable. It’s bad enough as is without every unemployed college graduate convinced that they can take this, make a small change and then try to sell it (creating, as well, an intransparent market, since no user of ratings will have the time or energy to analyze literally hundreds of rating suppliers to figure out who they should be buying this service from. Given the import of ratings for loan activity and portfolio management, this decreases market transparency, since there will be so many offerings that finding the ones done properly will become onerous).

    However, it is a start in demanding transparency and increasing understanding of how ratings actually work.

    1. LeeAnne

      they have defrauded and ruined the lives of tens of millions of people.

      Its not difficult to manipulate any number or rating on anything. therefore, in ‘rating’ agencies we have the unlimited power that corrupts unlimitedlyyyyyyyy

    2. Larry Barber

      The reason? The work involved is non-trivial. Seriously non-trivial, and making changes to methodologies can have far-reaching effects, ones that take a significant skill set to understand properly.

      Do you really thing that making an industrial strength operating system (RedHat, Debian, and so on), web servers (Apache, Tomcat, JBoss) or office suites (LibreOffice) is a trivial undertaking? That the people who make these systems don’t have significant skill sets? You sound just like a Microsoft mouthpiece circa 1995.

    3. Marc Joffe

      Thanks for your thoughtful comments, John, but I find myself disagreeing on a number of levels. First, I should reveal my bias: I am the creator of the open source Public Sector Credit Framework mentioned in Dr. Petrov’s article, so I clearly have an axe to grind here. That said, I have worked for a major “anglo saxon” rating agency and a couple of analytics vendors, so I know the other side.

      The rating process can be decomposed into a number of components, some of which can be open source and some of which can remain proproetary. In the case of PSCF, I have offered the community a multi-year fiscal simulation tool with a default point, which can be stated as a debt/GDP ratio or any other ratio you choose. PSCF is really just a piece of software plumbing – rather than a methodology. You could build sovereign and subsovereign default probability models in PSCF and keep them propritary. Alternatively, you could make the entire analysis transparent by publishing the inputs and outputs on the web.

      So a software tool like PSCF, or Intex for Structured Finance, or KMV CreditEdge for Corporates could be open sourced without forcing the rating process supported by such tools to become open source. Rating agencies, investors and analytic vendors could all contribute to and benefit from open source credit analytics to do their proprietary business.

      I also want to respond to your point about methodology changes. I don’t see the benefit to anyone other than the rating agency of keeping these confidential. With an open source methogdology, users would have access to a complete history of all the changes – due to the ability to duplicate and preserve content on the Internet. If the methodology contains an error, there are more eyes available to identify it and get it fixed. This process leads to better ratings, not “piss poor” ratings as you suggest. Also, the idea that armies of a unemployed college students will fork the methodology and create a morass of alternatives doesn’t seem plausible: why would an institutional investor pay any attention to them?

      1. John F. Opie

        @Marc Joffe: Illuminating comment. We are closer together than apart. Methodologies are changeable, but only under deliberate consideration, as they have negative real-world effects: no change in methodology, for instance, should lead to a major rating change unless there was something wrong, fundamentally, with the original methodology. Fundamentally, however, the problem is with acceptance within the customer base to methodology changes: changes here that impact, for instance, portfolio management decisions can literally cost (or save) millions. Change the methodology and the results on a constant basis and you will destroy any acceptance for ratings.

        @Larry Barber: Goodness, of course that work requires significant skills and is non-trivial: that was not my point and apologies if it was taken that way. My point, though, is that ratings can have real-world effects quite unlike those of operating systems, and perhaps a better simile is embedding OS for medical systems. If these are not carefully vetted and implemented, someone might die. That places rather different requirements for operating parameters and quality of code than the OS for an accounting computer running a payroll once a month. Ratings have the same impact for corporate life: given the impact of an incorrect rating for misallocation of capital – which should be the cardinal sin of any investment methodology – it can and does kill off companies and destroy jobs if not done correctly.

        A mixture of open-source and proprietary sounds good to me: the rating business desperately needs more transparency, not less.

  4. Stephen Nightingale

    (1) The Scientific Method predates the the Open Source model by about half a millennium. Linus himself credits the open Source success of Linux to the application of Scientific Method.

    (2) I never understood how S&P ‘downgrading’ US debt had any effect on the interest rates. Since the currency and the debt are sovereign, and Bernanke has been insistent that the Fed rate will stay low and stable at least through 2014.

  5. OMF

    The solution is to do away with ratings alrtogether. Make investors assess the bonds themselves, as they should have been doing all along.

    A rating is just another excuse for fund managers to hide behind after all.

    1. Marc Joffe

      You share that view with National University of Singapore’s Risk Management Institute. See http://rmicri.org/about/aboutcri.php. They argue that ratings are a public good.

      In the case of government bond ratings – which is my focus – it is hard for the government to provide this particular good. If the government issued ratings or controlled private entities that issued public sector ratings, the results would be open to skepticism because of perceived or actual conflicts of interest.

      I suggest to you that this is a job for an NGO.

  6. ScottS

    Even if the model is perfect, it’s still susceptible to garbage-in, garbage-out. We don’t prosecute for fraud, especially if it upsets the apple cart. Who will mandate that the entities being rated disclose the relevant data and punish them when they don’t?

  7. michael hudson

    Here’s the problem. It used to take me a year to do a single analytic chart of the US balance of payments; a year to do the oil industry’s accounts; a year to analyze a third world’s ability to pay.
    Only an enormously well-funded effort can do a serious analysis. Anything else is just opinion. So this is NOT like open source information technology. It requires working with companies, for instance, if you do a good analysis.
    Only the government is in a position to get the raw material. So the criticisms of the credit ratings agencies are all correct, the solution is utopian.

    1. Marc Joffe

      Michael: KMV’s modeled corporate default probabilities – updaed daily for 20,000+ companies – outperformed ratings in the 1990s. Moody’s reacted by buying the company for 4 times revenue. Models can outperform rating analysts without anyone talking to the company.

      Most governments publish their fiscal and economic data. Numerous companies, think tanks and academic institutions have the capability to analyze this data and offer an informed opinion about the issuers creditworthiness.

      This just isn’t that complicated. You can see my methodology and sample models at http://www.publicsectorcredit.org/pscf.html.

Comments are closed.