Quelle Surprise! Experts Give Self-Serving Diagnoses

New York Times writer David Leonhardt, in “When Trust in an Expert Is Unwise,” cites a small scale study of car mechanics that suggests that so-called experts often aren’t very skilled, and also too often prescribe inappropriate and costly remedies to simple problems.

Leonhardt suggests that the study’s findings can be generalized to fields like medicine, where the patient cannot readily judge a medical practitioners’ quality, and even if he has doubts in his recommendations, cannot readily challenge them either (most MDs are now very skilled when patients bring in printouts from the Interner).

The study used the comparatively simple standard of a car that had a few things wrong with it. Note that while all the mechanics were asked to do a thorough inspection, the results showed that few really did.

But consider an additional disheartening element: the car mechanics’ diagnosis was free. They got paid only for finding things wrong. But by contrast, in medicine, the law, and consulting, the diagnosis phase in charged to the client.

One can argue that that should provide good incentives, and it should be an improvement over a free examination. However, consider an additional fact: for all these professions, it is costly to find new customers. Thus there is an economic incentive to convert an analysis into a treatment. And of course, a more extensive diagnosis generates more revenues, whether that improves outcomes or not.

The worst is that many practitioners sincerely believe in overly elaborate initial exams, viewing them as “professional” as opposed to wasteful. And if this standard becomes established in an industry, it is very hard to combat it, even when there is no or limited objective support for the methodology.

From the New York Times:

A few years ago, an economics graduate student named Henry Schneider drove his dad’s old Subaru station wagon up to Montreal. He had heard about a Canadian consumer interest group that had done undercover investigations of auto-repair shops, and he wanted to try a more academic version of its experiment.

So when he arrived in Montreal, he handed the Subaru over to the mechanics working for the group, the Automobile Protection Association, for a complete inspection. They found that it had a small hole in its exhaust pipe, a blown taillight and several other relatively minor problems. Mr. Schneider took careful notes. But he also did something that no ordinary car owner would do. He asked the mechanics to show him how to mess up the car in a couple of serious but obvious ways.

They taught him how to loosen the battery cable (which can prevent a car from starting) and how to suck out coolant (which can leave an engine vulnerable to overheating). Armed with this knowledge, Mr. Schneider drove home to Connecticut and undertook a devilish little test.

Over the next few months, he took the Subaru to 40 garages, loosening the battery cable and draining some coolant before each visit. He even wrote himself a script and memorized it, to make sure he was telling every garage the same thing. “We bought the car recently, and we should have had it looked at before we bought it, but we didn’t,” he would say. “It hasn’t started a few times. Can you check that out?” He also asked for a thorough inspection.

Mr. Schneider was trying to answer a question that has occurred to pretty much all drivers who have ever been given the unsettling news that a car needs more repairs than they had expected: Does it really? Or is the garage just looking to make some extra money off me?

Economists sometimes refer to this situation as an “expert service problem,” because the same expert who is diagnosing the flaw is the one who will be paid to fix it…..

There is one school of thought — and it dominated the field of economics for decades — that says we shouldn’t worry so much about these problems. Even if most consumers can’t distinguish between good and bad expert service, some can. These informed consumers will tell their friends and relatives which experts to use, and an expert’s reputation will eventually become well known. The market, according to this view, solves the problem.

But there haven’t been too many real-world tests of the theory, which is precisely what makes Mr. Schneider’s research so interesting — and so depressing.

At only 27 of the 40 garages did mechanics tell Mr. Schneider that he had a disconnected battery cable, the very problem to which he had pointed them by saying his car didn’t always start. Only 11 mentioned the low coolant, a problem that can ruin a car’s engine. Ten of the garages, meanwhile, recommended costly repairs that were plainly unnecessary, like replacing the starter motor or the battery. (Tellingly, his results were in line with what the Automobile Protection Association found when it performed its experiments in Canada.)

In all, only about 20 percent of the garages deserved a passing grade. “And that’s with a pretty low bar,” Mr. Schneider told me. “I’m even allowing them to have missed a blown taillight that should have been caught.”….

Yet by studying a fairly narrow question, Mr. Schneider discovered a larger truth: the expert service problem really does appear to be a serious one…. when a situation is too complex for an amateur to grasp — and when it involves shades of gray — you probably shouldn’t expect to get a purely objective diagnosis from someone who has a financial incentive to give you something else.

That, in fact, may be the biggest flaw with today’s health care system. (Yes, even bigger than lack of health insurance.) If anything, Professor Hubbard argues that the expert service problem is more serious in medicine than in auto repair, because most people are less willing to question a doctor than to question a mechanic. Any effort to reform American medicine has to grapple with these conflicts of interest.

And what about your car? How can you be sure you’re not getting swindled?….

For an expensive repair, a second opinion makes sense, but it will be hard to know which garage to believe….Until some savvy entrepreneur starts a garage-rating business, the best solution may be the oldest one: asking for a recommendation from someone who is knowledgeable enough to distinguish between good service and bad. Just remember that a lot of people don’t know quite as much about cars — or their mechanic — as they think they do.