Reproducibility of experimental results is a hallmark of good science. Yet reproduction of results is rarely performed. There is little fame and often little external funding for being the confirmer, being second in the race. Hence the laurels in the Olympics of science are often awarded to those who publish first — or for the more practically minded — patent first.
It is therefore encouraging and stimulating when there is a serious effort to reproduce the results of others, as recently reported by Stuart et al in the JID. But as with all good things, there may be a curse — in this case, the “winner’s curse” should be considered.
The β-defensin gene often has multiple copies (up to 7-12), and previously, higher copy number of the defensin gene was associated with psoriasis. This might explain partially why psoriatic skin is rarely infected with pyogenic organisms. This finding was re-investigated with patients from both the US and in Europe, with particular attention to the technical details of determining the number of copies, increasing the number of patients and controls, and utilizing more highly powered statistics. After much analysis an increased prevalence of psoriasis was shown to be associated with increased copy numbers of the β-defensin gene. However, the odds ratio of the association were not as high as those reported in the initial study. This is not completely unexpected, since subsequent studies and meta-analyses of gene associations with complex diseases often show decreased odds ratios or even odds ratios without statistical significance . This is considered an example of the ”winner’s curse”. That curse requires some explanation.
The term originated in relation to activities such as sealed auctions or bidding on oil fields or baseball players, where the exact monetary value of the product is not known. The winning bid is often higher than that which a rational economist might predict. The statistical mechanisms of this phenomenon have been studied in detail and might be reviewed by those readers for whom standard sleeping pills are no longer effective. The odds ratio being higher than that which is estimated for future meta-analyses is the ”winner’s curse” for gene association studies. Since the winner often is awarded the next grant based on statistically significant initial data, this curse is one many researchers might welcome.
In addition to statistical concerns, known and uncontrollable heterogeneity in populations and heterogeneity of complex diseases such as psoriasis must be considered. I hope our mathematically and genetically inclined readers will take the opportunity to discuss “the curse” and share whether they worry about it affecting their research.
One must marvel that Mendel was smart enough to study single gene traits; however, he may have been the first victim of the curse: it could explain why his findings were undiscovered for decades.
Bazerman, MX and Samuelson, WF (1983) I won the auction but don’t want the prize. J. Conflict Resolution 27:618-634
Goring, HHH , Terwilliger, JD and Blangero J (2001) Large upward Bias in Estimation of locus-specific effects from Genomewide Scans. Am J. Hum Genet 69:1357-1369
Stuart PE, Hüffmeier U, Nair RP (2012) Association of β-Defensin Copy Number and Psoriasis in Three Cohorts of European Origin. J Invest Dermatol doi:10.1038/jid.2012.191