After an hour I realized, of course, that the time I'd wasted not buying the paper had been worth much more than the paper itself. So I bought the paper. Just this once.
The article in question is called "Stopping the Active Intervention: CARET," by Bowen et al., Controlled Clinical Trials 24 (2003) 39–50. As you might have guessed, I'm going to spend the rest of today's blog talking about it so you don't have to buy it yourself.
The paper I bought talks about the circumstances surrounding the halting of an infamous cancer-prevention study called CARET, otherwise known as the Carotene and Retinol Efficacy Trial. It was a large National Cancer Institute trial that didn't go well.
CARET was actually one of two large NCI vitamin studies that didn't go well in the 1980s. The other was the Alpha-Tocopherol and Beta-Carotene (ATBC) Lung Cancer Prevention Study. CARET was carried out in the U.S.; ATBC took place in Finland.
I'm not going to spend a lot of time talking about the studies here (for that, see my latest blog on this subject at Big Think). Suffice it to say that both studies began enrolling participants in 1985, the Finns at a rate of about 9,000 a year, the Americans at 2,000 a year. The ATBC study reached its enrollment goal of just under 30,000 people in three years. CARET went on enrolling for nine years, peaking at 18,314 people in September 1994.
It's customary in large trials to enroll people gradually—not just for practical reasons (it's infeasible to sign up 30,000 people at once) but also because if a treatment proves to be deadly, you want to be able to halt the study before huge numbers of people have been put at risk. Sadly, both ATBC and CARET put large groups at risk. Both should have been stopped prematurely. Only CARET was—and its halting came unnecessarily late.
The purpose of ATBC and CARET was to validate the usefulness of antioxidant supplements (Vitamins A and E, chiefly) as cancer-preventive agents. For ATBC, the study population consisted of male Finnish smokers aged 50 to 69. For CARET it was current and former smokers, male and female, aged 50 to 69, plus asbestos-exposed workers (all men) aged 45 to 74. (The asbestos workers made up 22% of the 18,314 study participants.) The experimental design in Finland was a 2x2 matrix design in which a quarter of the participants got 20 mg/day of beta-carotene, a quarter got 50 mg/day of alpha-tocopherol (Vitamin E), a quarter got both, and a quarter got placebo. Thus, treatment groups outnumbered placebo 3:1. In the U.S., CARET began with a 2x2 design but the design was changed early on so that the bulk of the study population got either placebo or a combo of 30 mg/day beta-carotene and 25,000 IU of retinol per day (thus a 1:1 ratio of treated to untreated populations).
According to the ATBC writeup that eventually appeared in The Journal of the National Cancer Institute, "An independent Data and Safety Monitoring Board convened twice annually to monitor trial progress and to study unblinded data that were relevant to intervention safety and efficacy." Which makes it all the harder to understand why the ATBC trial wasn't halted early when the beta-carotene groups separated from placebo—in the wrong direction. It was evident in Year 5 of the eight-year study that people in the treatment group were developing lung cancer at a heightened rate. In fact, that ended up being the key finding of the study: taking Vitamin E and beta-carotene leads to 18% more cancer.
Presumably, the Finns didn't halt their intervention early for the simple reason that the study's stopping criterion (whatever it happened to be; all large studies have them) wasn't met. Perhaps the divergence of treated and untreated group performance was deemed statistically non-significant. We'll never know for sure.
The ATBC results appeared in the April 14, 1996 issue of The New England Journal of Medicine. In "Stopping the Active Intervention: CARET" ($31.50 from Elsevier), we learn from the lead investigator and his coauthors:
On March 25, 1994, the NCI directed investigators of NCI-funded randomized trials involving beta-carotene to inform their data and safety monitoring boards of the ATBC trial findings, to review their data in light of these findings and to develop plans to inform their participants of the ATBC results and incorporate the findings in their consent forms.This is an interesting statement in a couple of ways. First, it doesn't say exactly who at NCI "directed investigators" to review their data. The implication is that somebody higher up (than the lead investigator) gave an order. Whoever that person was, he or she knew the unblinded Finnish results ahead of publication. Secondly, it says safety boards were directed to "review their data." But in the CARET study (unlike ATBC), the safety board did not have access to unblinded data. How can you review data that's still blinded? And if you did so, what purpose would it serve? You'd look at the data and see that one group of people, under one set of codes, was doing worse than another group under another set of codes. You might very reasonably assume that the group doing worse was the placebo group. But you wouldn't know for sure.
This is where it gets interesting, because in "Stopping the Active Intervention: CARET" ($31.50 from Elsevier) we learn that
On March 29 and April 1 [1994], CARET’s Principal Investigator convened telephone conference calls with the SEMC during which the committee was unblinded to intervention group assignment.Note the phrase "the committee was unblinded." In actuality, there was no unblinding over the phone. In a separate writeup in Data Monitoring in Clinical Trials: A Case Studies Approach (Springer, 2006), page 222, former CARET Safety Committee members Anthony B. Miller, Julie Buring, and O. Dale Williams explain that in August 1994—four months after the phone call—"we unanimously agreed that we should be unblinded as to the nature of the regimens given to the coded groups." The unblinding actually happened in August, after statisticians compiled their interim report for the Safety Committee—not March.
The account given by the lead investigators in "Stopping the Active Intervention: CARET" makes it sound as if urgent action was undertaken in March 1994 to begin a stand-down of the experiment. That's not what happened. Not by a long shot.
When the Safety Committee got its first look at unblinded data (in August 1994), there was no doubt that CARET's study population was experiencing the same elevated cancer rates seen by the Finns. The Safety Committee took a vote on whether to stop the CARET trial—and found itself deadlocked. Two members of the five-person committee were in favor of stopping CARET. The others thought it would be better to go ahead. The rationale for continuing in spite of elevated cancer in the treated group (here I quote from the account given in Data Monitoring in Clinical Trials) was:
- The statistical significance of the difference had not crossed the O’Brien–Fleming boundary (i.e., this could still be a chance finding).
- The effect was surprisingly rapid and must mean if real that preexisting (but undiagnosed) lung cancers had had their growth accelerated by the regimen.
- We knew of no mechanism of the action of beta-carotene that could have induced such an effect.
- There were other chemoprevention trials using beta carotene ongoing, to stop CARET now would have an undesirable adverse effect on these trials.
- We owed it to science to be absolutely certain of the adverse effect before stopping the trial.
- This was the second trial to show an adverse effect of beta-carotene chemoprevention; it was extremely unlikely to be due to chance.
- We owed it to the participants to prevent possible further harm to them. It was perhaps particularly unfortunate that the adverse effect appeared to be present in asbestos workers as well as current smokers.
- The adverse effects appeared not to be restricted to lung cancer; there appeared to be an adverse effect on cardiovascular disease as well.
In September 1995, a year after the first interim analysis, the Safety Committee got a look at updated results. They were even worse than before.
Many memos and meetings and conference calls and cross-country flights by NCI personnel later, a decision was finally reached in mid-December 1995 to pull the plug on CARET. The decision was approved on December 18. Then everyone adjourned for Christmas holidays. Then on January 12, 1996, letters went out (yes, by snail mail) to all CARET participants, informing them of the decision to end the study (and the reasons for the decision).
 |
| CARET found increases in all-cause mortality, cardiovascular mortality, and (not shown here) lung cancer mortality in high-risk individuals who took beta-carotene and retinol. (N Engl J Med 1996;334:1150-5.) |
When CARET's results were published in May 1996 in The New England Journal of Medicine, the medical world was aghast to learn that administration of beta-carotene and retinol to high-risk individuals actually increased the rate of lung cancer by 28% compared to placebo. Cardiovascular mortality, likewise, went up 26%.
The Finnish ATBC study had found an increase in lung cancer of 18%, based on a beta-carotene consumption of 20 mg/day. In the CARET study, participants had taken a 50% higher dose of beta-carotene (30 mg/day). They got 50% more cancer.
Hindsight is an evil game, and it's easy to bash NCI for not pulling the plug on CARET earlier than it did without knowing all the particulars. But frankly, what's to know? The Finns published their results in April 1994. Anyone could have looked at those results, then looked at the CARET experimental protocol (and study population), and immediately seen the red flags. Even without knowing the unblinded results, it would have been prudent to halt (or at least begin winding down) the study in April 1994, three and a half years ahead of the planned stop date. Instead, a befuddled Safety Committee (and a bureacracy-entrenched National Cancer Institute) waited until January 12, 1996 to send letters by first-class mail to people who were unnecessarily dying. In the irritatingly self-congratulatory account of this travesty in "Stopping the Active Intervention: CARET," we're constantly reminded that the study was terminated 21 months early, as if it's something to be proud of. The fact is, the study was terminated a minimum of two years later than it should have been, due to negligent disregard of the Finnish results (which were known ahead of time to NCI higher-ups). The Finns, too, should have stopped early, as soon as it became apparent that the treatment groups had diverged (in the wrong direction) from the placebo group, which is to say, in Year 5 of the 8-year study.
To argue that it was okay to wait an extra two years to stop the CARET study because the formal stopping criteria had not been met is a phony argument. When CARET finally was stopped, the stopping criterion had still not been met.
As I said before, I've provided more detail on ATBC and CARET (and other deadly vitamin trials) in a separate post at Big Think. Please read that post before deciding whether to adjust your daily vitamin routine. In the end, you might want to consider scaling back your use of Vitamins E and A if you're at high risk of cancer (and maybe even if you're not at high risk). The evidence says these supplements are harmful for at least some people. I'll post more on the subject here over the next few days.
Please share this story with your social media contacts if you found it helpful. And please see the companion post at http://bigthink.com/devil-in-the-data/the-dark-side-of-antioxidants. Thanks!
