Successfully reported this slideshow.
We use your LinkedIn profile and activity data to personalize ads and to show you more relevant ads. You can change your ad preferences anytime.
Upcoming SlideShare
What to Upload to SlideShare
Next
Download to read offline and view in fullscreen.

5

Share

The Peer Review Process

Download to read offline

Peer review process of academic publications is introduced and a sample given for how this is carried out within an academic teaching module. Guidance given on best ways to carry out a peer review.

The Peer Review Process

  1. 1. The Peer Review Process
  2. 2. The Academic Peer Review Process Peer Review in This Module Carrying Out a Detailed Peer Review
  3. 3. The Academic Peer Review Process
  4. 4. Paper is submitted to conference Paper Assigned to AC Sent to Reviewers Reviews Created Scores GivenNormally 3/4
  5. 5. AC Summary Created Paper Sent back to Author Rebuttal Period Rebuttal Submitted 2-Weeks Based on Reviews
  6. 6. Re-Scoring Accept/Reject Rebuttal Sent to Reviewers Can go up OR down
  7. 7. Submitted Accepted % Accepted 2014 2043 465 23% 2015 2120 486 23% 2016 2435 565 23% 2017 2400 600 25%
  8. 8. • Very detailed process with lots of people involved at each stage • Relies on the expertise within a group in order to judge whether a contribution is valid • Lets reviewers see the level that others are performing at and adjust their own effort levels
  9. 9. • The significance of the paper's contribution to HCI and the benefit that others can gain from the contribution: why do the contribution and benefits matter? Aspects to Cover
  10. 10. • The validity of the work presented: how confidently can researchers and practitioners use the results? Aspects to Cover
  11. 11. • Replicability: how easily would a researcher be able to carry out this work again based on the methodology discussion Aspects to Cover
  12. 12. • Presentation clarity; Aspects to Cover
  13. 13. • Relevant previous work: is prior work adequately reviewed? Aspects to Cover
  14. 14. ******* This paper is as good as any top paper in a conference/journal ****** This paper is as good as any standard paper in a conference/journal ***** This paper may appear in a conference/journal but I wouldn't argue for it to be there **** I don't like this paper but wouldn't object to others liking it *** I would rather not see this paper in a conference/journal ** I would argue for this paper to not be in a conference/journal * I would strongly argue for this paper to not be in a conference/ journal
  15. 15. ******* This paper is as good as any top paper in a conference/journal ****** This paper is as good as any standard paper in a conference/journal ***** This paper may appear in a conference/journal but I wouldn't argue for it to be there **** I don't like this paper but wouldn't object to others liking it *** I would rather not see this paper in a conference/journal ** I would argue for this paper to not be in a conference/journal * I would strongly argue for this paper to not be in a conference/ journal Average Cut Off Point
  16. 16. Peer Review in This Module
  17. 17. Paper is submitted to conference Paper Assigned to AC Sent to Reviewers Reviews Created AC Summary Created Paper Sent back to Author Rebuttal Period Rebuttal Submitted Re-Scoring Accept/Reject Rebuttal Sent to Reviewers Paper Given to Mike Sent to Peers/ Classmates Mike Makes Comments Grade Awarded
  18. 18. • Gives you an opportunity to critically evaluate your peers work and to comment on strengths and weaknesses • Allows you to understand the level of quality that is present within the class • Presents you with an opportunity to learn from others work
  19. 19. Carrying Out a Detailed Peer Review
  20. 20. The First Read Through The first read-through is a skim-read. It will help you form an initial impression of the paper and get a sense of whether your eventual recommendation will be to accept or reject the paper. 1. What is the main question addressed by the research? Is it relevant and interesting? 2. How original is the topic? What does it add to the subject area compared with other published material? 3. Is the paper well written? Is the text clear and easy to read? 4. Are the conclusions consistent with the evidence and arguments presented? Do they address the main question posed? 5. If the author is disagreeing significantly with the current academic consensus, do they have a substantial case? If not, what would be required to make their case credible? 6. If the paper includes tables or figures, what do they add to the paper? Do they aid understanding or are they superfluous?
  21. 21. Spotting Major Flaws While you should read the whole paper, making the right choice of what to read first can save time by flagging major problems early on. 1. Drawing a conclusion that is contradicted by the author's own statistical or qualitative evidence 2. The use of a discredited method 3. Ignoring a process that is known to have a strong influence on the area under study
  22. 22. Spotting Major Flaws If methodology is less of an issue, it's often a good idea to look at the data tables, figures or images first. Especially in science research, it's all about the information gathered. 1. Insufficient data 2. Statistically non-significant variations 3. Unclear data tables 4. Contradictory data that either are not self-consistent or disagree with the conclusions 5. Confirmatory data that adds little, if anything, to current understanding - unless strong arguments for such repetition are made
  23. 23. Concluding the First Reading After the initial read and using your notes, including those of any major flaws you found, draft the first two paragraphs of your review - the first summarising the research question addressed and the second the contribution of the work.
  24. 24. Concluding the First Reading This should state the main question addressed by the research and summarise the goals, approaches, and conclusions of the paper. It should: First Paragraph 1. Help the editor properly contextualise the research and add weight to your judgement 2. Show the author what key messages are conveyed to the reader, so they can be sure they are achieving what they set out to do 3. Focus on successful aspects of the paper so the author gets a sense of what they've done well
  25. 25. Concluding the First Reading This should provide a conceptual overview of the contribution of the research. So consider: Second Paragraph 1. Is the paper's premise interesting and important? 2. Are the methods used appropriate? 3. Do the data support the conclusions?
  26. 26. Second Read Through The Introduction and Literature Review A well-written introduction: 1. Sets out the argument 2. Summarises recent research related to the topic 3. Highlights gaps in current understanding or conflicts in current knowledge 4. Establishes the originality of the research aims by demonstrating the need for investigations in the topic area 5. Gives a clear idea of the target readership, why the research was carried out and the novelty and topicality of the manuscript
  27. 27. Second Read Through Methodology Academic research should be replicable, repeatable and robust - and follow best practice. Methodology should give enough detail so that other researchers are able to carry out the same research. For example, equipment used or sampling methods should all be described in detail so that others could follow the same steps. Where methods are not detailed enough, it's usual to ask for the methods section to be revised.
  28. 28. Second Read Through Results and Discussion This section should tell a coherent story - What happened? What was discovered or confirmed? Discussion should always, at some point, gather all the information together into a single whole. Authors should describe and discuss the overall story formed. If there are gaps or inconsistencies in the story, they should address these and suggest ways future research might confirm the findings or take the research forward.
  29. 29. Second Read Through Results and Discussion Certain patterns of good reporting need to be followed by the author: 1. They should start by describing in simple terms what the data show 2. Once described, they should evaluate the trends observed and explain the significance of the results to wider understanding. This can only be done by referencing published research 3. The outcome should be a critical analysis of the data collected
  30. 30. Second Read Through Conclusions This section is usually no more than a few paragraphs and may be presented as part of the results and discussion, or in a separate section. The conclusions should reflect upon the aims - whether they were achieved or not - and, just like the aims, should not be surprising. If the conclusions are not evidence-based, it's appropriate to ask for them to be re-written.
  31. 31. Second Read Through References You will need to check referencing for accuracy, adequacy and balance. • Accuracy - Where a cited article is central to the author's argument, you should check the accuracy and format of the reference - and bear in mind different subject areas may use citations differently. • Adequacy - Are important parts of the argument poorly supported? Are there published studies that show similar or dissimilar trends that should be discussed? • Balance - Check for a list that is helpful for the reader, doesn’t rely overly on self-citations.
  32. 32. Structuring Your Report Structure your evaluation of the paper into three sections: • Summary • Major Issues • Minor Issues
  33. 33. Structuring Your Report Summary • Give positive feedback first. Authors are more likely to read your review if you do so. But don't overdo it if you will be recommending rejection • Briefly summarise what the paper is about and what the findings are • Try to put the findings of the paper into the context of the existing literature and current knowledge • Indicate the significance of the work and if it is novel or mainly confirmatory • Indicate the work's strengths, its quality and completeness • State any major flaws or weaknesses and note any special considerations. For example, if previously held theories are being overlooked
  34. 34. Structuring Your Report Major Flaws • Are there any major flaws? State what they are and what the severity of their impact is on the paper • Has similar work already been published without the authors acknowledging this? • Are the authors presenting findings that challenge current thinking? Is the evidence they present strong enough to prove their case? Have they cited all the relevant work that would contradict their thinking and addressed it appropriately? • Are there any major presentational problems? Are figures & tables, language and manuscript structure all clear enough for you to accurately assess the work? • Are there any ethical issues? If you are unsure it may be better to disclose these in the confidential comments section
  35. 35. Structuring Your Report Minor Flaws • Are there places where meaning is ambiguous? How can this be corrected? • Are the correct references cited? If not, which should be cited instead/also? Are citations excessive, limited, or biased? • Are there any factual, numerical or unit errors? If so, what are they? • Are all tables and figures appropriate, sufficient, and correctly labelled? If not, say which are not
  36. 36. The Academic Peer Review Process Peer Review in This Module Carrying Out a Detailed Peer Review
  • NainRajwanshi

    Aug. 31, 2021
  • GilchristDsouza3

    May. 24, 2021
  • LaibaAmin1

    Jul. 3, 2018
  • MateenAkhtar4

    Mar. 18, 2018
  • mauriciodemedeiros1

    Mar. 12, 2018

Peer review process of academic publications is introduced and a sample given for how this is carried out within an academic teaching module. Guidance given on best ways to carry out a peer review.

Views

Total views

1,441

On Slideshare

0

From embeds

0

Number of embeds

2

Actions

Downloads

42

Shares

0

Comments

0

Likes

5

×