How to peer review an academic paper

Academic journals are dependent on peer reviews to publish high-quality articles. PhD students can greatly benefit from becoming peer reviewers. Read about how you can become a peer reviewer, how to write a review and the dos and don’ts of providing feedback on manuscripts for academic journals.

Contents

Why PhD students are often the best peer reviewers

The first time you will receive an email with the subject “Invitation to Review for the Journal of…” is very exciting. At the same time, your imposter syndrome might kick in. Am I good enough? I am not an expert in this field. Who am I to criticize a potentially well-established scholar?

Journals, however, depend on so-called peer reviews. And journal editors know that PhD students often are the best reviewers. They take the task seriously. And they put a considerable amount of work into a review. Additionally, article reviews on your academic CVs are an advantage. It displays academic involvement and expertise. But let’s start from the beginning.

How to become a peer reviewer for an academic journal

Are you a PhD student interested in gaining experience as an article reviewer? There are several ways to get started.

For instance, consider submitting your own research to journals and filling out a profile section with keywords that represent your research interests. This could lead to being contacted by the journal to review someone else’s submission.

Another strategy is to leverage your network, particularly your PhD supervisors. Many sit on editorial boards of journals or know people who are journal editors. Let them know that you are interested in reviewing a paper and they might be able to make it happen.

For more concrete strategies, check out my separate post on 5 proven ways to become an academic peer reviewer.

Receiving and accepting/rejecting an article review request

Once you have a review request sitting in your mailbox, it becomes real. Most review requests include the paper abstract. Read it carefully. Imposter syndrome is one thing, but you may have no clue about the topic at hand. If that is the case, kindly reject the review.

If the abstract addresses a topic that is connected to your PhD, or your field of study, you can accept it. You don’t need to be an expert on the specific subtopic. However, you should be familiar with some broader disciplinary discussions and publication standards.

Before pressing the “accept” button, check the timeline. Most articles need to be reviewed within 4-6 weeks. Once you accept, you commit to submitting your review on time. You won’t get a fine if you don’t make the deadline, but many journals have a scoring system of reviewers in the background as well. So if your reviewer profile is marked as submitted late, you are less likely to receive a new review request.

A typical article review request looks like this:

How to write an article review

Once you accept a review request, you usually receive a link to a full paper manuscript. The first step is to read the paper. What helps is to already open a text document to write down your first impressions. I often copy-paste certain sentences and note the page number and my reaction. Think of “I don’t get this”, “Why?!”, “This requires further info”. (This will be re-written properly later).

A good article review is critical but constructive. Instead of stating, for example, that the theoretical framework is messy, provide suggestions on how to improve it. You can also provide specific literature suggestions that might improve the paper.

Coming to the most important point: be kind! You can be critical without attacking someone. Academia can be a toxic place, and mental health problems are skyrocketing. Receiving an extremely harsh review feels like a punch to the gut. It can make the receiver question his or her intellectual ability, and research in general.

Don’t be part of this toxic culture. You don’t have to bring others down to flourish yourself! So don’t write anything that you wouldn’t say to someone’s face. You could write:

“The theoretical framework is weak, unconvincing and does not make any contributions to the field.”

But you can also write:

“I struggled to see the red thread in the theoretical argument, as the author tries to include too much. I see a lot of potential in enhancing our understanding of xxx. Therefore, I would suggest that the author rewrites the section with a more focused aim in mind, which will also increase the paper’s contribution to this specific set of literature.“

Which type of feedback would you prefer..?

Elements of a good academic article peer review

You read the paper, you wrote down your feedback and now you need to finalize your review. Next, get organized. A good article review is not too short and not too long. Think of 1 to 1.5 pages.

A good way to write a clear review is to categorize your comments. You can work with the following headlines:

• Abstract (Is it reflecting the content of the paper? Is it clear and concise?)

• Introduction (Are the topic, the societal and scientific relevance well introduced? Is there a clear argument presented in the introduction?)

• Literature review and theoretical framework (Is the paper embedded in the recent international literature on the topic? Does the author make a compelling theoretical argument?)

• Methodology (Are the methods of data collection and analysis justified? Does the selected methodology fit to the research question and aims?)

• Analysis (Are rich empirical results presented to support the author’s arguments? Does the analysis sufficiently link to the literature review and theoretical framework?)

• Discussion and conclusion (Is the author critically reflecting on the findings? Is the empirical analysis strongly linked to existing theory? Are the contributions of the paper highlighted, and are there suggestions for future research?)

• Academic writing (Is the writing easy to follow? Is the narrative straightforward? Are there grammar or language mistakes? Are sources cited correctly?)

• Other comments (Anything else you might want to share, such as whether you think the author selected the right journal for this publication).

You see that just writing one or two paragraphs for each point will easily result in 1.5 pages. If it is a bit longer, no worries. But don’t write an essay. Furthermore, remember that you are not an editor. If you notice many grammar mistakes, you can mention them and recommend language editing. However, you do not have to list every single grammar mistake. That is not the job of a reviewer.

Submitting a manuscript review

To submit a review, you will usually also receive a link. Some journals ask the reviewer to simply upload the review document or to copy-paste the text somewhere. Others ask some more detailed guiding questions. The most important one is whether you recommend to

i) accept the manuscript without revisions

ii) request minor revisions

iii) request major revisions

iv) request ‘revise and resubmit’

v) reject the manuscript.

Accepting a manuscript without requesting any changes rarely happens. Rejecting a manuscript can happen when you don’t see much potential in the paper. Or maybe you think that it simply does not fit into the journal. If so, again, be kind. Suggest a different journal, saying it might be a better fit.

If the feedback involves smaller issues here and there, you should go for minor revisions. If you would like to see, for example, the whole theoretical framework to be rewritten, you select major revisions.

Journal editors make a final decision based on 2-3 reviews of a paper. In cases in which the reviewers disagree (e.g. one says reject, and the other says accept), editors often read the paper thoroughly as well. And then decide.

Very often, after the author completes revisions, the editor will try to reach out to the same reviewers for another round of peer reviews.

Getting recognition for completed article reviews

Article reviews are unpaid. Thus, they are part of the free labour that is expected within academia. At the same time, it is good for your CV. It is recommended to do several reviews (think 3 or 4) every year.

As a PhD student, it is a great opportunity to learn and do reviews. It trains you to reflect on your work, as you know what reviewers are looking for. As a result, your papers get better as well. It also forces you to read literature that is maybe not directly connected to your thesis topic.

All in all, doing peer reviews as PhD students is recommended. Particularly for students that aim for an academic career. But also those who want to leave academia can benefit from the experience, and become better writers themselves.