Following my tweets on the Kokko & Wong (2007) retraction story which generated a fair amount of interest, Retraction Watch invited me to blog about my role as a reviewer in this saga:
“The decision by the journal Evolution to retract a 2007 paper after one of the authors corrected a mistake in a later paper has hit a nerve, with many academics (myself included) taking to Twitter to support the authors and criticize the journal’s handling of the case (the story was first covered on Retraction Watch here). As a reviewer of the later paper, and indeed, the person who requested that the original paper be corrected, I felt it was important to publicly express my opinion on the matter.
The story started in 2007, when Hanna Kokko and Bob Wong published a theoretical paper in Evolution about sex roles and mate searching. Several years later, Kokko (University of Zurich, Switzerland) decided to revisit the topic with Lutz Fromhage (University of Jyväskylä, Finland) and Michael Jennions (Australian National University). Fromhage spotted key mistakes in the original analysis of the model, which meant that Kokko and Wong had incorrectly concluded that the evolution of mate searching is not influenced by the unavailability of females (e.g. due to pregnancy).
The analytical error led Kokko and Wong to suggest that the reason males are usually the searching sex is instead due to polyandry. The corrected analysis, included in the later paper, contradicted these conclusions: time spent unavailable for mating does weaken selection for mate searching, and so it is not polyandry per sethat causes male-biased searching but rather the presence of a mating window during which the timing of mating does not matter for females.
In fall 2015, I was asked to review the new paper, which highlighted these problems with the original paper. I concluded that the new study was solid, well-written and that I would be happy to see it published in Evolution. I did, however, raise the following concern:
The manuscript corrects significant technical problems with [an] earlier paper in this journal (Kokko and Wong, 2007). These problems need to be highlighted at source (i.e. as corrections/erratum next to the original paper), with readers directed to the new paper for a fuller exploration of the corrected model.
I believe this would have been a reasonable course of action, as it would have ensured that readers of the 2007 paper would be aware of the errors without purging the paper from the scientific record. At no stage did I feel that a retraction would be appropriate.
In the decision letter the Editor and Editor-in-Chief agreed that:
it would be most appropriate to publish a separate erratum to the previous paper. This could then be linked to both the previous paper and the current [manuscript]. In that case this current paper would not have to change the way it is handling the previous paper much at all.
At this stage, my role in the review process ended and I was unaware of the outcome until I saw the Retraction Watch article earlier this week. I’ve since discovered that the authors were unable to publish an Erratum on the grounds that the author list was not identical to the original paper. Fromhage et al. therefore submitted a Technical Comment to Evolution to highlight the problems with the original paper at the source. I was not invited to review the Technical Comment, which was rejected by another reviewer and the Associate Editor, both of whom recommended retraction of the original paper. The journal then retracted the original article and published the new paper. The decision as to who to invite to review is obviously up to the journal, but I think it was a little odd in this case seeing as I was the person who had requested the original article be corrected.
So why have I spoken out about this case? The problems of post-publication peer review have already been highlighted elsewhere, and it certainly isn’t rare for a paper to be retracted due to an honest mistake (although most retractions are due to misconduct). Moreover, one could argue that the mistakes in Kokko and Wong’s 2007 paper were sufficient to warrant a retraction as they significantly affected the conclusions. But by that logic, a large number of empirical studies should also be retracted due to incorrect statistical analyses or overreliance on fickle p-values, leading to irreproducible results.
My concern is that the forced retraction of the original paper sends a bad message to the scientific community. Kokko has effectively been penalized for critiquing her own work, when in fact she should be applauded for her honesty. We should be encouraging others to emulate Kokko’s approach, but I fear that Evolution’s decision will only serve to deter such behavior. In an era when funding decisions and faculty positions depend so heavily on publications, it is asking much of scientists to put aside self-interest and be openly critical of their own work if this risks damaging their future prospects.
The most important thing for science is that mistakes in the literature are identified and resolved as quickly as possible, which is why I suggested the problem be corrected at the source. Publication does not grant some special truth status; we should always critique the literature and not assume that results or conclusions are true simply because they have undergone peer-review. However, highlighting and correcting errors does not necessitate airbrushing the scientific record.
In the case of Kokko and Wong (2007), does a retraction achieve anything that an Erratum or Technical Comment could not? In my opinion, no. The editors were no doubt in a difficult position; they often need to err on the side of caution and the merits of a retraction in this particular case could be argued either way. But their decision has implications beyond this case and risks dissuading people from openly addressing their mistakes. Forced retraction is unavoidably associated with a certain degree of stigma, given its association with misconduct (although there was no suggestion of misconduct in this case, simply an honest mistake). Forcing a retraction on the authors therefore feels like a punishment, which is probably why this story has resonated with so many people.
Scientists are not perfect. Honest mistakes will happen along the way, but it’s important that we encourage people to admit when they have gone wrong, and as one Twitter user put it: “not make doing what is good for science bad for scientists.”