Wednesday, January 07, 2015

Winner's Curse

Before saying anything more about genetic studies, I want to talk about a problem that's quite common in scientific studies. It's colloquially (and somewhat inappropriately, since the analogy doesn't hold 100%) known as the "winner's curse."

In economics, the idea of "winner's curse" comes from the observation that the winner of an auction often overpays. Maybe you've noticed this phenomenon yourself, if you've participated in auctions? It's not just that people get into a frenzy of bidding and drive prices too high. That's not really the core idea here. The core idea is more like this: Suppose a laptop goes on auction and there are ten people in the room. All will bid on it. Every one of the ten persons has an estimate (a top bid) in mind for the true value of the laptop. The average of those ten estimates might be (let's say) $300. However, an average bid doesn't win an auction, does it? The highest bid does. And that might be considerably in excess of $300.

The "winner's curse" phenomenon comes into play in science a lot, because (for example) when a systematic review is done for various studies that found the "effect size" for a given type of medical treatment, the effect size often turns out to be larger in smaller studies. In theory, a treatment should give roughly the same effect size in all different sizes of study. If aspirin is a successful treatment for headaches 50% of the time, a study involving 30 people should show that, and a study involving 1000 people should show that. Instead, what you often see is (here I'm making numbers up) aspirin works 60% of the time in a small study and 50% of the time in a large study.

You can see how this might happen. Suppose four different teams of scientists (who don't talk to each other) decide to investigate the effectiveness of a vitamin milkshake as a hangover cure. And suppose that the "true effectiveness" of this "cure" (over a large enough number of studies and subjects) is actually about 20%. However, our four teams, working with very small study populations (hence, a high potential for statistical noise) find effectiveness of 11%, 13%, 22%, and 30%. The teams that got the lowest numbers probably won't publish their results. The team that got 30% probably will.

This sort of thing happens in science a lot and helps explain why, for example, some of the early twins studies on schizophrenia found high concordance rates (over 60%) for schizophrenia in identical twins versus fraternal twins, in relatively small studies, whereas later, much larger studies have found rates as low as 11%.

"Winner's curse" is a well-known ascertainment problem in science that affects studies of many kinds, including some of the recent large genome-wide associataion studies that have produced findings that failed to replicate when other teams decided to do similar sorts of investigations. I'll be talking more about that about soon.

Note: For a more technical discussion of "winner's curse" in genetic studies and what can be done about it, see Sebastian Zöllner and Jonathan K. Pritchard, "Overcoming the Winner’s Curse: Estimating Penetrance Parameters from Case-Control Data," Am J Hum Genet. Apr 2007; 80(4): 605–615.

I'm working on a no-nonsense, "skewer the sacred cows" book about mental illness. If you'd like to follow the progress of the book, sign up for the mailing list, and be sure to check back here regularly. Thanks!

3 comments:

  1. Our company has well equipped machines and modern tools that ensure quality and fast production with the help of which we are able to manufacture more than 5,000 office chairs per year. And thus, we are able to meet the ever- growing demand of chairs. Among the many places where our products are supplied among those two biggest supplies are done in Mumbai, Pune in India.
    Chair Manufacturers in Mumbai
    Chair Supplier in Mumbai
    Office Chair Supplier in Mumbai
    Visitor Chair Supplier in Mumbai
    Chair Dealers in Mumbai
    Top Chair Manufacturers in Mumbai

    ReplyDelete
  2. Are you looking for Packers Movers in Mumbai. Then you have just reached right website of Packers and Movers Services in Mumbai. Our service as Packers and Movers in Mumbai is rated as Top Packers and Movers in Mumbai. When shifting of households and private vehicle comes, its often to critical to choose reliable Packers and Movers in Mumbai. As smooth and safe delivery of your goods is our prime objective.
    Please Visit Our Website :http://apackersmovers.com/
    Movers and packers in Dhanu Road
    Movers and packers in Mumbai CST
    Movers and packers in Masjid
    Movers and packers in Sandhurst Road
    Movers and packers in Byculla

    ReplyDelete
  3. Indian Packers and Movers in Mumbai is the best packing and moving company in Mumbai. Of course, the credit goes to our hardworking team in Mumbai and world class infrastructure. Customer satisfaction is a great asset for the company. We appreciate the feedback of our clients that ultimately help us to improve. Indian Packers and Movers in Mumbai have a long list of reputed clients who assign their projects to relocate. We offer our customers all-in-one packing and moving experience with suitable & affordable moving and relocation or packers movers services.
    Packers and Movers in Colaba
    Packers and Movers in Cuffe Parade
    Packers and movers in CST
    Packers and movers in Mumbai central
    Packers and movers in nepean sea road
    Packers and movers in malabar hill

    ReplyDelete

Add a comment. Registration required because trolls.