Blind Dates in Sciences
Dealing With Rejection in Peer Review
Every researcher remembers the moment when he or she sends in a manuscript or proposal for peer review. It is truly like going out on a blind date. Without knowing the identities of the peers, we often wait in the dark for an answer as to whether our best ideas will receive precious funding or years of hard labor will be recognized by a prestigious journal. Because our academic success and sometimes even viability depend on such an outcome, the wait can be excruciating. Unfortunately, as any seasoned member of the scientific community will warn you, the outcome of scientific peer reviews is dominated by rejection. Current funding levels at the National Institutes of Health imply that ≈90% of applications will be rejected and some top tier journals see even higher rejection rates among submitted manuscripts. Learning the odds-improving skills to navigate through various peer review processes has become a required course of scientific training and teaching curriculum for junior investigators.1 However, no matter how much you think you are prepared, the reality is, as long as we choose to pursue a scientific career, rejection from peer review will always be a constant in this occupation. Therefore, it is useful to discuss how to make the peer review experience constructive rather than destructive. In this article, we will focus our discussion on the peer review process for manuscripts, with particular insights provided by the editors of several leading scientific journals.
With overwhelmingly negative verdicts and being blinded from much of the process, an author’s perception of peer review is expectedly dark and suspicious. Based on an online survey reported by Ho et al,2 less than half of authors think that biomedical journal review systems are fair and only a quarter of them think that the outcome is scientifically justified. Many authors view the peer review process as an antagonistic battle between us and them, although almost all of the peer reviewers are actually members of the same scientific community. Understandably, the most common response to a rejection is often emotional. Authors frequently file quick rebuttals to protest the verdict in an angry, defensive, and suspicious tone based on the assumption that the negative reviewers were incompetent, erroneous, or biased because of conflict of interest. “This is a frequent mistake,” Dr David Eisner, Editor-in-Chief for the Journal of Molecular and Cellular Cardiology, notes. In fact, he and Dr Howard Rockman, Editor-in-Chief for the Journal of Clinical Investigation, share a surprising observation that, more often than not, the accused biased reviewer was actually the one recommended by the authors (presumably the buddy reviewer), although some of the most positive evaluations came from reviewers on the request to be excluded list. Similar advice was echoed by Dr Eric Olson, Editor for the Proceedings of National Academy of Sciences and Circulation Research, about rebuttal letters: “Don’t be personal and accusatory.” “A response to the reviewers that is written in a tone that is annoyed, condescending, or worse still, abrasive, is almost invariably doomed to failure and almost invariably leads to rejection,” adds Dr Roberto Bolli, Editor-in-Chief of Circulation Research. Clearly, a universal mistake many authors have made in response to rejection is an emotional and quick protest without rational discussion, which will accomplish nothing but a firm rejection.
In fact, the peer review system is a fundamental pillar for scientific advancement that continues to evolve along with professionalization and specialization of scientific enterprise itself.3 The process is conducted under the premise that fair and objective evaluation by peer experts is the best guarantee that meritorious and significant findings can be published with a seal of approval.4 Blinded peer review, although continuously debated for its pros and cons, remains the most accepted practice because anonymity permits unobstructed expression of opinions (in both constructive and destructive senses).5 Funding agencies, such as National Institutes of Health and American Heart Association, and major scientific journals are all making major efforts to improve the fairness and transparency of the peer review process and to minimize potential conflicts of interest.6 It is important to appreciate that constructive critiques are essential elements of scientific progress where innovative ideas and concepts are reinforced and solidified by rigorous testing and by overcoming healthy skeptics. Unbiased critiques from an experienced investigator can be extremely helpful in revealing previously unrecognized weaknesses of the study. A constructive review can help authors in many ways. For examples, reviewers can identify major weaknesses in experimental design and data interpretation that need to be addressed to enhance the validity of the findings. Reviewers can help better analyze the significance of the finding based on past literature that authors might have missed. A rigorous analysis of the methodology from a more experienced reviewer may identify better and alternative approaches to add support to the conclusions, and reviewers can improve the presentation of the study by picking out misstatements, wrong labels, and grammatical mistakes in the manuscript. Finally, a fresh look from a new angle may motivate additional thinking and provoke new ideas for future studies. As a result of the peer review process, a report is often much improved and stronger and can endure better the test of time and has more effect on the field in a long run. Indeed, when peer review works, the process should not impede scientific progress but rather help to accelerate research and foster future successes.7 So, after cooling off the emotional steam on receiving the bad news, let us discuss the best steps to move forward.
For a scientific report, the factors determining where it can be published have a lot to do with its content and matching the interest of the readership to a particular journal. Although many journals use impact and novelty as a guideline for acceptance, there are significant differences in priorities and focuses among them. Therefore, it is important to investigate the target journals and evaluate your choices not simply based on their prevailing impact factors but whether your study is a good fit.
“Prepare the cover letter well with clear articulation about the main findings of your work, but never exaggerate,” is a common message from several journal editors who commented on this subject. “Deliver a focused message clear and loud,” suggests Dr Rockman. Because novelty and impact are usually the most significant elements in scientific evaluation, authors should emphasize these aspects with both clarity and care in cover letter and abstract. Factual representation of your findings relative to the state of the art in the field is generally better than decorating with extravagant words Authors should provide a clear argument for the novelty of the study based on facts, whether it is conceptual (new ideas never proposed), mechanistic (new interaction/player never implicated), or methodological (new tools/system important for the field), and offer an assessment as to why it is not an incremental advance relative to the current state-of-the-art. Dr Michael Basson, a senior Editor for Nature Medicine notes, “A claim of novelty by dismissing prior literature,” will not be viewed kindly by the editors or reviewers. To argue for the potential effect of your study, the authors should provide a clear argument about how the study advances the field: for example, how the new concept will change the current direction or practice in the field, how the new findings will propel more studies in the future, or how the new method will be used by other investigators in the field. Once a journal is selected, it is important to adhere to the specific format requirements and to ensure consistency in text and figure layouts. Last, considering the observations made by editors about biased reviewers, it is not necessarily wise to recommend only your buddies as potential reviewers.
Evaluation of a Rejection
Rejections often come in different forms and with underlying messages. The most promising ones show a great deal of enthusiasm (where resubmission or revision is encouraged) with clearly identified guidance for resubmission. Others contain a significant reservation while explicating major issues that demand significant additional experiments to address (de novo resubmission with no guarantee of eventual acceptance). Clearly, rejections with uniformly negative comments suggest that a more drastic action (such as submitting to a different journal) would be necessary. The comments from the reviewers or the editors often reveal the underlying reasons for a rejection. Novelty, significance, conclusions, methods, data quality, and presentation style can all affect the final decision. Although we know reviewers may not be always right, finding the solution to address the reviewers’ concerns is the best way to move forward.8 Sharing a rejection letter is not fun, but seeking input from your colleagues and mentors can be extremely helpful. Consulting with experienced investigators is a useful step for formulating proper responses.
It is important to view these critiques with a cool head and an open mind. First of all, evaluate the validity of the verdict from the reviewers and editors based on a factual analysis of the arguments. If the progress is built on a previous study, the concept may not be as novel as you have thought. If a critical piece of data is missing, the central hypothesis may not be as sound as it should be. If the methods need to be scrutinized more with additional positive and negative controls, these improvements may be necessary to support the claimed conclusions. It is important to evaluate all issues, big or small, and to design a strategy of response. Authors need to balance the suggested experiments (wish list from the reviewers) with their scientific validity and practical feasibility. As mentioned below, these deliberations will be helpful in winning a good argument for a rebuttal or resubmission.
Rebuttal and Response Statement
Although rejection is common, success in some form after revisions is also the norm. If authors decide to resubmit a revised manuscript, the response letter/statement offers a great opportunity to exchange your views with the reviewers and editors. It can be crucial to the eventual outcome.
The rebuttal letter should be factual, courteous, and nonconfrontational. “Thanking the reviewer for his criticism is always an excellent idea. Pointing out the time and effort that the reviewer has spent in the review is likely to generate good will,” recommends Dr Bolli.
Listing all the issues in detail and categorizing them as major versus minor and fixable versus nonfixable would be a good step in preparing your responses. According to Dr Olson, it is not good practice to “selectively skip the tough points,” but it is good to “repeat the same response to each reviewer when an issue was raised by multiple reviewers.” Dr Bolli has a similar advice for response statements. “Authors should copy verbatim each statement contained in the reviewer’s critique, in bold letters, and then include his/her response underneath in regular fonts. This should be done with great detail and precision.” Therefore, addressing all issues instead of evading them is the best response strategy.
It is ok to disagree with a reviewer’s comments and not to follow all the suggestions as long as the justification is based on factual arguments and clear logic. Mixed opinions among different reviewers is common. Leveraging positive comments against negative ones among the different reviewers is a plausible strategy. Authors should stay on the issues and use data to support his/her arguments when communicating with reviewers. If particular experimental data cannot be provided as requested, “state clearly the reason why and, preferably, indicate how many dollars, how many months, and how many animals would be required to provide these data,” suggests Dr Bolli. The authors should keep in mind that the response statement is a precious channel of communication with the reviewers and the editors. Just as in the manuscript itself, the best argument can only be made with data and facts not empty statements. Your message can be most effectively delivered if the manner of its delivery is professional, measured, and respectful.
Dissent and Consensus
The peer review process can be obstructive when an innovative concept is rejected because of lack of consensus among the reviewers. In fact, each groundbreaking new idea must contradict a current paradigm and faces the risk of rejection by even the most competent and earnest reviewers. Fortunately, scientific journals and funding agencies are making efforts to address this issue. New funding mechanisms such as the Director’s Award at National Institutes of Health and Innovative Research Grant at American Heart Association are designed to support innovative research proposals with emphasis on innovation, out-of-the-box ideas, and paradigm-shifting concepts. Recently, the journal Antioxidants & Redox Signaling experimented with a so-called Rebound Peer Review mechanism for papers rejected during normal peer review.9 Authors whose work was rejected could choose to identify reviewers to advocate openly for the acceptance of the article. As scientific research evolves, the peer review process will also continue to innovate and adapt to serve the ultimate interest of the communities.
As Dr Olson observed, “All important work will be published eventually, and you can’t win them all.” Publishing your findings is not the final goal but only an essential step for us to advance our science. Making the peer review process a constructive one with the help of new insights and fresh perspectives from peers is essential to improve our own research and to the long-term vitality of scientific research. In addition to being a successful author, one thing we can all contribute to ensure a fair and effective peer review system is to participate actively as a reviewer and be a good one.1 As eloquently pointed out by Dr Thomas Annesley in a recent article, “there are many reasons we can argue against serving as a reviewer,10 but none of them will stand if we appreciate the critical importance of a healthy peer review process to the well-being of our community and ourselves.”
I thank Dr Åsa B. Gustafsson for the opportunity to present this assay in the Early Career Development Workshop at the 2013 BCVS Scientific Conference, Las Vegas. I am personally in debt to Dr Michael Basson, Dr Roberto Bolli, Dr David Eisner, Dr Eric Olson, and Dr Howard Rockman for sharing with me their insights and advices.
The opinions expressed in this article are not necessarily those of the editors or of the American Heart Association.
- © 2014 American Heart Association, Inc.