The Life Scientists’ Guide to Peer Reviewing Scientific Papers
Reviewing your first manuscript for a journal can be both an exciting and daunting experience. You might be wondering where to start, what aspects to critique, or struggling with imposter syndrome. In which case, the best way to get started is to “learn-by-doing” and pair up with a more experienced mentor who is willing to guide you through the process and provide constructive feedback.
As early-career researchers ourselves, we wanted to share what we have learnt from our experiences, and help the next generation of research scientists write clear and constructive peer reviews.
Why peer review?
Your first thought on reading this blog may be that peer-reviewing for journals is going to be time-consuming. However, before you start running a cost-benefit analysis in your mind about whether or not it’s a valuable activity to be involved in during your PhD and early post-doctoral years, here’s why we peer review:
- It’s a rewarding process to facilitate authors in presenting their most compelling evidence (both positive and negative results) and contributes to advancing a field of science
- We believe that, as research scientists, we have a duty to contribute to the system we use to have our own manuscripts reviewed and published
- Peer review is not only a significant contribution to the scientific community, it’s often an important requirement for promotion
- It is necessary for scientists from diverse backgrounds and experiences to review so that there is a sufficient breadth of knowledge to assess the interpretations of the results and offer alternative explanations
- Critiquing others’ work helps you identify weaknesses in your own manuscripts, in turn strengthening your work
- You gain insights into work outside of your immediate field
- You learn experimental design features for techniques outside your skillset
- It can be an important first step to becoming involved with journal editing, a great way to build your reputation in your field and expand your research network
How do you become a reviewer?
It’s unlikely that you’ll be approached by journal editors to peer review until you have published several first-author or corresponding author papers. However, there are strategies you can put in place to get your foot in the door and start to build up your experience:
- Your supervisor/mentor probably receives more requests to peer review than are feasible for them. If you let them know that you’re interested in learning to peer review, this is an organic way to access manuscripts and mentor-mentee support. Ensure that when your reviewer’s comments are submitted back to the journal that you are named as a contributor to the work to help establish your profile as a peer reviewer at that journal. Some journals even have a designated section for recognition of ‘reviewers-in-training’.
- Enter a mentor program specifically designed to educate PhD students and postdocs in the process of constructive peer review. There are several offered by societies, journals, or publishers, such as the SfN Reviewer Mentor Program for the Journal of Neuroscience.
- Join Publons and create a public profile, which tracks and verifies not only your publications but also your peer review and editorial contributions. As a member of Publons, you will have access to Publons Academy, which offers the Learn to Peer Review with Confidence course. As part of this course, you will gain practical experience, get mentorship, and on completion of the course you will be a certified peer reviewer and you can use this status to connect with journal editors in your field.
Getting started with peer review
Once equipped with a manuscript, mentor, and the naturally analytical mind common to scientists, it’s time to get started!
To help, we’ve compiled the following checklist based on what we have found useful to guide us through successfully generating peer reviews.
20-question peer review checklist
Ask yourself the following 20 questions:
- Do you have sufficient time and expertise in the field of the manuscript to be making an educated review? If you aren’t confident with a particular aspect of a manuscript, it’s ok not to comment on it, though you should make this clear to the editor in your confidential remarks.
- Do you understand the scope of the journal? This is important for determining if the manuscript fits the readership of the journal and for the level of scrutiny required when reviewing.
- How does the journal want the review to be formatted? Read all the material provided by the journal about what they would like you to consider while reviewing.
- What questions do you have? As you read the manuscript, make note of what questions come to mind – are they answered/discussed by the authors? What don’t you understand?
- Where are your knowledge gaps? What are the concepts you may need to learn, or refresh your memory on, to be able to conduct this review?
- Do I need to talk to experts? Ask questions of your local experts to better understand any concepts or methodology you are unfamiliar with. Remember though, it is important to make sure that unpublished work remains confidential as there could be an unknown connection between your local expert and the authors.
- Does the TITLE adequately reflect the contents of the manuscript?
- Is the ABSTRACT informative and accurate?
- Are the research AIMS clearly stated?
- Are the METHODS appropriate and are they adequately described?
- Do the EXPERIMENTAL APPROACHES serve the study’s purpose? Are the experiments designed such that they can answer the question posed? Have the appropriate controls been used? Is the N number high enough (has a power calculation been done?) and do the N numbers make sense throughout the data being presented?
- Are there any ETHICAL ISSUES concerning animal experiments or human subjects?
- Are the RESULTS clearly described?
- Is the DATA PRESENTATION sufficient? Do the graphs make sense, are all axes visible, do the images show what they claim to show? Are the figure legends appropriate with sufficient detail?
- Have the appropriate STATISTICAL tests been performed and are they reported correctly?
- In the DISCUSSION, are the conclusions and interpretations sound and justified by the data? Do the data answer the research question, or is there over-stating of results or only causal links between evidence and conclusions? Are there alternative interpretations of the data that have been missed by the authors?
- Is the LANGUAGE in the manuscript well written, concise and logically structured? Are the number of figures/tables appropriate?
- Is there evidence of plagiarism? Check the references, especially in the methods section, to confirm correct citation of protocols. Make use of plagiarism detection software if they are available to you (e.g. iThenticate, TurnItIn).
- Is there evidence of falsified data? Have a sharp eye for immunoblots (have they been cropped?), immunohistochemistry or immunocytochemistry microscopy images (increase the image contrast to look for image manipulation) and graphs (does the data look too good to be true?). If you think you may have detected falsified data, check first with your mentor or a more senior researcher to see if they support your conclusion then contact the editor to raise your concern.
- Is the referencing appropriate and recent? Have the authors missed any prominent papers relevant to the research?
In the process of working through this checklist, you should read the manuscript at least 2-3 times.
It can be beneficial to give the paper a thorough read-through, wait a couple of days, and then revisit again with fresh eyes – it’s amazing what you may miss on the first couple of reads.
Presenting your review
It is important to keep in mind that your overall goal is to facilitate the authors to present the most accurate interpretation of their data.
The authors can’t read your mind or get in touch with you quickly to ask: “What do you mean by this?”, so keep your feedback clear and concise.
Although the format of peer review might vary from journal-to-journal, the convention is to provide a summary and a list of the major and minor flaws.
How to present a peer review
- Make a SUMMARY based on your interpretations of the results (not the authors’ interpretations). Do you think this work is:
- Novel?
- Of interest to the journal’s readership?
- Has the potential to be widely cited?
- An incremental or significant advancement to the field?
- Organise your points into MAJOR FLAWS that need to be addressed. Major flaws usually constitute problems with experimental design or lack of experimental evidence to support the conclusions. The most helpful way to highlight a major flaw is to say what is wrong, why is it wrong, and then provide an example of how it can be resolved. For example:
- What is wrong: “You shouldn’t use a t-test on this data.”
- What is wrong + why it is wrong: “You shouldn’t use a t-test on this data because the data points are not normally distributed.”
- What is wrong + why it is wrong + how to resolve the flaw: “You shouldn’t use a t-test on the data because the data points are not normally distributed. A Mann-Whitney U test should have been used.”
- Next, list the MINOR FLAWS. This might include toning down the overstating of results or conclusions, a list of missing labels, inconsistent use of terminology or abbreviations, suggested changes to figures for clarity, etc.
- The final component when presenting your review (and one which is often overlooked) is your TONE OF VOICE. The first version of the review might read as curt and exasperated, because that may be how you feel! However, reviews should always be edited before submission to address the tone of voice. Chances are you’ve had your work rejected and it hurts. Break the cycle and be constructive, yet nice, as this could be years of someone’s life, or an entire PhD project! Think about how you would like to receive the feedback. It takes 5 minutes to do this final step and is good practice since some journals are moving towards a transparent peer-review process by publishing the reviewer’s comments alongside the paper. For example:
- First version of the review (unedited for tone of voice): “You shouldn’t use a t-test on the data because the data points are not normally distributed. A Mann-Whitney U test should have been used.”
- Review edited to improve the tone of voice: “The statistical analysis could be improved by using a Mann-Whitney U test as the data points are not normally distributed.”
Remember to consider and comment on the strengths of a manuscript as well, not just its weaknesses. If the authors have done something good, let them know!
Completing your review
Once you have completed your review, you should have a good gauge on the quality of the manuscript. Editors will often ask you to make a recommendation as to whether you think the manuscript is acceptable, acceptable with minor revision, needs major revision or should be rejected.
Although the final call lays with the editor, your viewpoint will be taken seriously. Consider the following when making your recommendation:
- Acceptable: The manuscript is good to go, congratulate the authors on a job well done!
- Acceptable with minor revision: Overall, the authors have provided a convincing amount of evidence to support their hypothesis. There are a few corrections to be made however these are more ‘minor flaws’ than ‘major flaws’.
- Needs major revision: There are fundamental flaws that can be addressed by following the reviewer’s recommendations.
- Rejected: Fundamental flaws with the manuscript that cannot be corrected.
Final tips when starting out with peer reviewing
- Make sure you have your work checked, or that the review is conducted in collaboration with your mentor until you are confident to conduct the peer review alone
- Keep a record of your review so you can go back to it, either for when the second round of revisions come back or for future reference when you review your next paper
- If possible, read the other reviewers’ comments. This is a good way to gauge how you are getting on – what did you miss, what did you agree on, how thorough were they?
- If possible, link your review to your ORCiD account and Publons to start building up your track record
- It’s ok to turn down invitations if you don’t feel qualified (though be sure this isn’t imposter syndrome at work) or if you don’t have the time. In these instances, pay the opportunity forward and make one or two suggestions of colleagues who you think would be qualified and could benefit from the experience. Science will suffer if you do a rush job or are way out of your depth.
- You will get faster – your first paper will probably be your longest as you are building your reviewing skill set. The more you do, the more apt you will become at reviewing.
- Everyone will have a different reviewing style, so experiment until you find what works for you – this is just a guide to get you started.
_________________________________________________
We wish you all the best on your peer review journey!
If you have any follow-up questions to this guide, feel free to email Rachelle Balez (rb478@uowmail.edu.au) or Rebecca San Gil (r.sangil@uq.edu.au).
And if you have any great tips to add to this blog, please post them in the comments below!
_________________________________________________
Additional resources
- Peer Review, the nuts and bolts, A guide for early career researchers: https://senseaboutscience.org/wp-content/uploads/2...
- Springer Nature: Resources for Peer Reviewers: https://www.springernature.com/in/reviewers
- Nature Masterclass: Focus on Peer Review: https://masterclasses.nature.com/focus-on-peer-rev...
- BMC Peer Review: Tips for Junior Researchers: https://www.biomedcentral.com/collections/peerrevi...
About the authors
Rachelle Balez is a senior PhD candidate at the Illawarra Health and Medical Research Institute, University of Wollongong, whose research focuses on using induced pluripotent stem cell models to investigate Alzheimer’s disease. Rachelle enjoys the opportunity peer review gives her to deep dive into new science outside of her immediate research field. Rachelle has reviewed 18 manuscripts under the guidance of several colleagues; however, she still remembers the feeling of imposters syndrome when she started peer-reviewing and hopes this guide will support others on their journey.
Dr Rebecca San Gil is the Fight MND Early Career Research Fellow in the Neurodegeneration Pathobiology Laboratory at the Queensland Brain Institute, whose research focuses on identifying novel therapeutic targets for the treatment of motor neuron disease. She is two years post-PhD and has reviewed seven manuscripts at international journals under the mentorship of her supervisor Dr Adam Walker. Self-confessed newbie at reviewing papers, Rebecca is excited to share the tips and tricks that she has learnt along the way.
_________________________________________________
If you enjoyed this article, why not check out the other resources available on our blog. We are passionate about supporting life scientists, early career life scientists and PhD students - with really low- priced reagents and biochemicals, grants and resources to help with both personal and professional development. We know how tough it is - so we hope you find these helpful!
Advice & guidance for life scientists
Click below to view our essential guides and articles to support life scientists, PhD students & early career life scientists:
Wellbeing for scientists
Click below for our resources to help improve your wellbeing:
Technical resources
Try our Molarity Calculator: a quick and easy way to calculate the mass, volume or concentration required for making a solution.
Try our Dilution Calculator: an easy way to work out how to dilute stock solutions of known concentrations
And - when you get to the stage of planning your experiments, don't forget that we offer a range of agonists, antagonists, inhibitors, activators, antibodies and fluorescent tools at up to half the price of other suppliers - click below to see how we compare with other suppliers:
