PEER assessment – advantages and caveats

My university has begun a trial using a peer assessment tool, so I thought it would be a good idea to explore the research around its efficacy, and how and if it should be utilised. Here are some of my thoughts on the pedagogy.

Peer assessment or peer review has two elements: viewing others’ work, and feeding back on what is seen. Both have their own pros and cons that should be considered when employing them as learning strategies.

Viewing others’ work

Viewing others’ work presents numerous pedagogical advantages. Foremost of these is the chance for a student to evaluate their own thinking on a topic compared to what their peers are presenting. It is in this phase of comparison that the active student is forced to decide if they are indeed on the right track or need to adjust their thinking. Such a learning opportunity is enhanced further in two ways. Firstly, a student having produced a piece of work themselves and likely to have read it over several times may be unable to see how it can be improved or if it contains errors, having become affected by the principle of inattentional blindness. Seeing the same assessment articulated in another way can mitigate against such a phenomenon. Secondly, repeated exposure to the content at hand contributes, as Nuthall suggests, to improved processing of content and its likely transfer into the long-term memory, an essential process for it to be of any use in a future learning context.

Another advantage is that seeing others’ insights into a topic can generate/stimulate the learner’s own creative exploration of the topic. This idea applies not just to the evaluation of content, but also to the evaluation of structure and general writing/presentation skill. Seeing models of strong writing and well-structured assessment adds to the viewer’s schema of such a context. If the peer review is set up so the student can then adjust their work before a final submission, then the resultant production is likely to be significantly better. If the review is only to be done after the final submission (but still necessary as part of the assessment), viewing others’ work still provides the advantage of formative learning, strengthening understanding and potentially providing the student with feedback before they receive their grade. Considering that most students tend to not read feedback once the grade is eventually seen, peer review may be the only opportunity to close learning gaps and assist future learning once assessment is submitted. If they see that their submission is vastly inferior to others’, the hyper-correction effect may come into play, where egregious errors tend to be re-learnt with more strength. Ohlsson’s Resubsumption theory however tempers such a likelihood, as explained succinctly by Greg Ashman.

The main disadvantage of the process is the likelihood of the expertise reversal effect occurring with students who read others’ work that is well below their level of understanding. The experience not only becomes redundant, but potentially regresses the student in their learning. Tudge (1989) explores such a notion and uses Vygotsky’s Zone of Proximal Development to suggest that development is not unidirectional; once placed in a new social context the student is at its mercy. Their progression is dictated by a movement towards the average between the two peers: perfect for the weaker student who improves, worse for the stronger student whose progress is vitiated.

Further disadvantages include the amount of time it takes to view several other pieces of work, potentially reducing motivation in the pedagogy, and if the peer review is undertaken before the final submission the chance of a student copying others’ ideas and not learning in the process; not actually doing any evaluation, possibly because they haven’t contributed any initial thinking to begin with. However, some technology can diminish this likelihood as the student is not able to see others’ work until they have submitted their work, and students can notify the teacher if someone has submitted nonsense just to activate the viewing of others’ work.

Feeding back

Feeding back may be comprised of two elements: applying a rubric or criteria, and providing comments.

Having to apply a rubric to grade or provide feedback to another student forces the student to pay close attention to the demands of the rubric, and by doing so potentially increases their learning to follow a rubric closely when producing their next piece of assessment. The close following of instructions of guidelines or advice or plan is certainly a favourable graduate attribute. However, being able to successfully apply a rubric is certainly no easy feat, and it takes considerable practice and indeed expertise to be able to so. The reliability issues, stemming from the subjective nature of most criterion and then in applying it too, in getting a consistent awarding of grades is well known, and this is for experts who have done so many times. Doud et al (p69) expose this further stating that often the assessments designated for peer assessment are actually the hardest ones to assess, even for the expert/instructor, containing overly subjective elements, like group work and oral assessment. Providing a model of how the criteria have been applied to an assessment is essential to begin the training of students in both reading and applying a rubric. It will go some way to mitigate the reliability issue, but it is always going to be a concern.

As stated above, there is a tendency for the stronger student to regress in peer review, but such a situation can be alleviated if the stronger peer takes on the role of the teacher and is willing to offer constructive advice to the other. This however, requires some training, or at least some explicit modelling, as it should not be a given that the more knowledgeable peer is able to successfully communicate their superior knowledge; many may be familiar with this phenomenon in higher education.  

Interestingly too, Tudge describes that student confidence also plays a major role in the success of peer tutoring. He suggests that even the weaker peer still will not get much out of the experience if the more knowledgeable peer wasn’t confident in communicating their knowledge. This, as well as the difference in ability levels, has large implications for how students are assigned in peer reviewing. The randomisation of assignment may in fact not be the wisest strategy, a fact reiterated by Whitman. In a course with hundreds of students, a solution to this may be that you manually assign those you know are high achieving to each other, and let the algorithm sort the rest.

Providing quality feedback in the form of comments is another skill in itself, and students will need to see models of how this is done to see what type of comments can be of benefit to the assessee, and the expectations around length and tone of comments. It becomes both an opportunity to teach students about constructive feedback, and also about leadership. However, there is quite a bit of evidence to suggest that the giving of feedback (the person who views others’ work) gains a lot more out of the process than the receiver of peer feedback.

Recommendations

1. Make explicit your motivations for peer review to students, and the benefits that you believe will be afforded to their participation.
2. If possible, deliberately assign who reviews who.
3. Consider how many other students each reviewer has to review – this may be inversely proportional to the size of the task
4. Begin with a small task first, helping students to build their skill and understanding of peer review. This will inevitably involve two aspects: reviewing small pieces of assessment, and not large essays, and having students only apply 1 or 2 criterion to the review.
5. Provide models and examples of what a good and bad one looks like so students can be assured they are on the right track and not waste energy which could reduce motivation.
6. Be realistic with how much time you will have to spend ensuring that the validity of the peer reviews is appropriate. Weigh up how many times will you have to go in and check it is being done fairly and well with the overall pedagogical benefits to students. Students will lose motivation if they see the feedback is poor or inaccurate.
7. Discuss with students the outcomes of the pedagogy, and consider ways the next time you do it could improve.

References

Boud, D., Cohen, R., Sampson, J. (2001). Peer Learning and Assessment. In D.Boud, R. Cohen, J. Sampson (Ed), Peer Learning in Higher Education. Kogan Page Limited, London.

Tudge, J. (1990). Vygotsky, the zone of proximal development, and peer collaboration: Implications for classroom practice. In L. C. Moll (Ed), Vygotsky and Education. Instructional implications and Applications of Sociohistorical Psychology (pp155-172). Cambridge University Press

Whitman, N. 2001. Peer Teaching in a School of Medicine. In N. Falchikov (Ed), Learning Together, Peer Tutoring in Higher Education (p265-9). Routledge Falmer, London.

I’m Paul Moss. I’m a learning designer at The University of Adelaide. I’m on Twitter too