The Leucht paper can hardly be considered unbiased, since the lead author received fees for consulting and/or lectures from Bristol-Myers Squibb, Actelion, Sanofi-Aventis, Eli Lilly, Essex Pharma, AstraZeneca, MedAvante, Alkermes, Janssen/Johnson & Johnson, Lundbeck Institute and Pfizer (and grant support from Eli Lilly), while another author received fees for consulting and/or lectures from Janssen-Cilag, Sanofi-Aventis, Johnson & Johnson, Pfizer, Bristol-Myers Squibb, AstraZeneca, Lundbeck, Novartis and Eli Lilly. Still, it's an interesting paper and deserves to be read in detail.
 |
| SMD (a proxy for effect size) for studies of non-psychiatric medications (white circles) versus psychiatric medications (filled circles, right). |
What the authors basically tried to show is that if you transform a big bunch of heterogeneous studies into SMD form (mostly using "responder" data, not remission or cure rates), and don't put confidence intervals on the dots, you find that psychotropic drugs are, on the whole, not inferior to other drugs.
However, if you dig into the data (see Table 2 of the Leucht paper), you find that psychiatric drugs have the strongest data in a comparison of drug versus placebo for relapse rates rather than primary treatment effects. The highest blue dots on the graph are from this kind of data. The fact that people relapse in huge numbers when they withdraw from drugs is thus shown in the graph as a strength for drugs, when in fact it's arguably a weakness. (Relapse rates are known to be higher for drugs, for example, than for psychotherapy.) But even if you don't back out some of the high points, response rates for antidepressants are generally unimpressive, with the ratio of drug response to placebo response being 53.3% to 42.4% in one case, 46.0% to 31.0% in another case, and 37.8 to 24.2% in the third of the three analyses looked at. When you combine this with the fact that 18% to 36% of drug responders eventually relapse, the overall picture can hardly be described as encouraging.
I was also struck by the fact that response rates to antipsychotics, for schizophrenia patients, is quite unexpectedly poor. Here, the only two studies cited (which were by Leucht and colleagues) showed a response rate of 40.6% for second-generation antipsychotics (versus 23.7% for placebo) and 29.3% for Haldol (versus 19.5% for placebo). The low response rates here may reflect Leucht's (admirable) tendency to rely on more recent meta-analyses. Early (and generally smaller) meta-analyses of nearly all the psychiatric drugs showed much larger effect sizes than more recent studies.
Speaking of placebo, the Leucht review offers a striking demonstration of the power of the placebo effect, showing, as it does, that placebo response is significant even for diabetes (21.7% for metformin, the only drug reported), stroke (55.8%), breast cancer (42.4%), and antibiotics (22.2% to 25.7%). No doubt spontaneous remissions account for some of the cures reported for "placebo"; but the same is true in mental illness (which is notoriously episodic, regardless of category).
It would have been quite telling if Leucht had produced a graph (similar to the one above) showing only placebo response rates. The graph would look quite similar to the one they produced (albeit with dots shifted slightly lower). Such a graph would demonstrate that placebos for psychiatric trials are non-inferior to placebos for other drug trials.
Of course, one problem with analyses of the Leucht kind is that they're 100% biased in favor of published results. We aren't seeing results from studies that showed no efficacy, because such studies aren't generally pubished very often. This is an important omission, because we know from the work of Kirsch and others that FDA is sitting on a ton of clinical trials that showed safety, but not efficacy, for a variety of psychiatric drugs; indeed, this was a major finding of the Kirsch meta-analysis (see link at the top), which drew from published and unpublished FDA trials obtained by Freedom of Information Act request. Kirsch found that half of antidepressant trials submitted to FDA failed to establish meaningful separation from placebo (and those trials rarely got published, or if they did, their data was conflated with data from more successful trials). To learn how drug companies hide data and spin "lack of efficacy" into efficacy, read the excellent paper by Spielman and Parry (2009), "From Evidence-Based Medicine to Marketing-Based Medicine: Evidence from Internal Industry Documents."
Leucht et al. claim, with their paper, to have put psychiatric drugs in the proper medical perspective. To some extent, this is true. On the whole, though, it doesn't change the fact that drugs for serious mental illness (schizophrenia, bipolar, depression, etc.) are ineffective (producing no response, or even a negative response) for the majority of people who try them, a result that's clear from Table 2 of the Leucht paper.
I am 99% done writing a mental health book (112,000 words) that's part science, part memoir. It's not an anti-psychiatry screed, although it's (appropriately) critical of many aspects of the profession. The 275 footnotes will let you decide which parts are more (or less) believable than other parts. To learn more about the book, add your name to the mailing list and return here often. Thank you!
