Headagogy 10 - Peer Assessment Ep3
Thu, Oct 20, 2022 8:56AM • 25:57
SUMMARY KEYWORDS
students, peer assessment, power, grade, assessment, soft skills, arthur, assessed, excalibur, study, peers, exercised, important, methods, authoritarian, learning, discourse, knight, developed, hear
SPEAKERS
Steve Pearlman
Steve Pearlman 00:02
And moving into this final phase of our discussion of peer assessment and engraving, it seems appropriate to close with some of the research that affirms the value of peer assessment. Now, I should note, of course, that real gains in critical thinking always require actual instruction. But there is mounting evidence that merely engaging in peer assessment itself actually does a great deal for developing soft skills, including critical thinking. I think it's really worth noting that increasingly, we're seeing calls from major international organizations talking about how critical it is for Education to develop soft skills. UNESCO in 2018 mentioned things like empathy, leadership, responsibility, integrity, self management, sociability, decision making, the World Economic Forum and 2020 mentioned soft skills, including analytical thinking, active learning, complex, problem solving, leadership, initiative, resilience, reasoning. Well, all of these things are things that happen in peer assessment, and I'll show you some of the research on that in a second. But all these things are happening in peer assessment, and in many other on grading methods, as well many other progressive or alternative grading methods. But let's just take a moment to establish a little contrast, if we take something like peer assessment where students are involved, they're active participants, they are having to think they're having to socialize, they're having to communicate, they're having to be resilient because their work is being assessed in front of them. They're increasing their capacity to self assess and to develop themselves based on how they are assessed to change, to evolve, to work together to resolve conflict, when there's disagreement about a grade that should be given. All of these things are happening in peer assessment. If we contrast that against traditional assessment forms that are entirely non interactive, that merely turn students into passive vessels, and that in no way cultivate any of those skills that we are finding valuable. Why would we want to use traditional assessment methods that do nothing for students other than tell them where they stand on scales, they often don't understand when we could just as easily and more profitably in other ways, work with alternative assessment methods. And by the way, study after study that I'm not even going to bother to list for you note that students favor alternative assessments, any alternative assessments where they get to reflect more where they get to be involved more where they get to be more active, they always say that those assessments are more valuable for them. I want to speak directly to one research study that was done recently, learners perceptions on peer assessment in team based learning classroom that appeared in the Learn journal language education acquisition Research Network in 2021. And it surveyed students as to their experience with peer assessment. And they put students on a Likert scale between strongly agree and strongly disagree with respect to a number of different questions. I'm not going to give you the exact breakdowns for every single one of these statements. But I'm just going to say that on the whole, the responses were largely, if not entirely, within strongly agree and agree, and I'll read a number of the statements for which there was majority consensus or vast majority consensus, if not unanimous consensus by the students, I became more involved and responsible in the class activities, I could better reflect on my personal performance and my role and responsibilities as a team member, I developed better evaluation skills, I became more engaged and responsible for my own learning, I learned to give more relevant and practical feedback to my peers. I improve my critical thinking skills through peer assessment activities. I've developed better communication skills when giving and taking feedback and discussing comments with others. I learned how to effectively discuss and negotiate about the teamwork procedures. I learned to adjust my learning approaches. I learned to be fair and unbiased and giving evaluation and feedback on others performance. Every student can benefit from peer assessment regardless of their learning competence. Peer assessment helps us reflect on our performance. Peer assessment helps us clearly understand and evaluate our roles and responsibilities in the team peer assessment made it possible for students to support each other and give help when it was needed. And there are others. But I think you're getting the trend here. And this is by no means the only study that affirms similar results with respect to peer assessment. The fact is that students typically just find it valuable. But I want to be careful here about something and putting COVID Circumstances aside, there is movement around the notion of using computer mediated peer assessment where students can respond to each other online rather than doing this exercise in person. But we don't want students to hide behind their computers, either when giving feedback or receiving it, we have to cultivate human beings to be able to interact with receive feedback from discuss ideas with disagree with resolve conflict with other human beings, they have to be able to do that in person with one another. I'm not saying that computer mediated peer assessment is always inherently problematic. I'm just saying that I think we have to be careful of placating students concerns or fears and insecurities about that. And instead, I think we need to do the opposite, we need to move them to places where they can be more resilient in these contexts, and face their peers and get through it. I can't tell you the number of students who after their peer assessment experience, especially those who go very early, if not first, in the process, they say, Wow, that wasn't bad at all. In fact, that was really positive for me. And they face their fears. And they faced a group of people who gave them commentary. And they got through it not as a negative experience through which they had to grit their teeth. But they learned to embrace this. And very often, by the time we are a few weeks into the semester, we have students who are saying, I'm excited to get my peer assessment, I want to hear the commentary and critique of my peers, I want them not to hold back. And often, even though they're typically not allowed to talk as authors through most of that session, as I typically run it, many students end up saying things like, don't hold back, everyone don't worry about hurting my feelings, I want to hear what you have to say. So while we might need to invoke computer mediated methods, at times, I hope it's something that we don't give into if we can avoid it. Obviously, for those of you teaching classes that are online, it's unavoidable. And it's better to do it online than not do it at all. There's also some very interesting evidence that suggests that high functioning students, those more capable in the classes feel as though they get a little bit more out of peer assessment than students who feel as though they're not as capable or are struggling a little bit. That's not very conclusive. The scale tilts a little bit, let's put it that way, depending upon the study. But that hasn't been my experience. And I think it has a lot to do with how well we work with students and train them in the process. The more we're leaving students to their own accord, initially, the less secure they feel in their capacity to use that peer assessment practice to advance themselves. So naturally, students who feel more capable to begin with are also still going to feel as though they are improving well, and doing well in the course, students who are more insecure to begin with may not have that insecurity buffered by the peer assessment experience as much as they like. But again, that hasn't been my experience, because I think it's really important to integrate that assessment experience into the practice of the course. But here's something even more interesting and probably more important for our purposes. Study after study affirms of finding that students actually feel as though they gain greater critical thinking skills, and sometimes greater understanding of what to do in their own writing, not when they are the author being assessed, but when they are the ones doing the assessment of other students. Because the process of generating the assessment and the process of reasoning through a grade reasoning through the criteria, and the process, most importantly, of reasoning through how to advise another student as to how to do better is a very high functioning moment for them, as opposed to just hearing the things that they need to alter in their own writing. And, of course, something that I always do with students, maybe not every session, but certainly every semester several times, is I have them write a reflection about how grading their peers has affected their understanding of what they need to go back and do with their own writing. So we're bridging that gap between the advice they're giving to others and what they're reflecting on and taking action about in their own work. And if courts, I think it's very important to note that through a number of different studies dating back almost 30 years now, though, I think this is the most difficult thing to measure, students actually perform better and learn more through peer assessment than through traditional assessment methods, we find greater achievement in their writing. And we find greater understanding, of course content through this process of peer assessment than through traditional assessment. And very often, depending on the study, often more so than self assessment. In fact, one of the most recent meta analyses done on this, it was done in 2020. It's published in educational psychology review, titled The impact of peer assessment on academic performance, a meta analysis of control group studies, the author's looked at studies dating back to 1995. And they studied this across different educational levels, primary, secondary, tertiary, they did it across a range of disciplines, accounting, education, medicine, politics, reading science, writing, I'm just naming a few. And they looked at in different forms, including whether was written anonymous, free form, dialogue, and what have you. And overall, what they found was that peer assessment is, in fact, a statistically significant improvement over no assessment. And over traditional assessment mechanisms. It wasn't always over self assessment, which obviously involves some degree of reflection, which is very important. But all the studies certainly did not reflect the depth of experience that I'm advocating for here, in which we really work with training students, so that they become acculturated to it and become fluent in it and become colleagues. Many of the studies that they looked at involved much more limited interventions of peer assessment, and nevertheless found a positive effect on learning outcomes. But let's say for a moment, just a moment that peer assessment didn't improve learning outcomes, that it didn't hurt learning outcomes, but it didn't improve learning outcomes. They remained steady, whether peer assessment was involved or not, which isn't the case. But let's just say that was the case. But nevertheless, we developed in students that critically important skill sets for our democracy in our society of resilience and discourse, of not passively receiving authoritarian evaluation and authoritarian truths, of taking active roles in their own learning in resolving conflict and developing all of those soft skills that we talked about, it's almost possible to say, Who gives a shit if it doesn't do much for their educational outcomes if it's still doing all of those other things. But in addition to all of those soft skills, in addition to all of those ethical things we are doing with students, in addition to how we are empowering them to contend with power, it actually does improve learning outcomes. And that is why it's such an excellent method of engraving, it makes them less focused on the grade and more focused on their work. And it sends the very clear message that there is a role for them as valued, able, thoughtful members of our intellectual community, at our educational institutions, not disenfranchised members whose voices are not really heard. But members who actually can contribute not in a way that is necessarily equal to faculty members in all regards, and the students don't care about that, but in ways that are nevertheless meaningful and impactful. And since I forgot to mention this earlier, I'd add one other wonderful benefit of this grade complaints go way down, they almost disappear. And the contentiousness between ourselves and our students also disappears, because we are no longer the ones assigning a grade to them. We are no longer the one sitting on Mount Olympus making decrees about their grades and about their competencies. Once they understand the criteria and what we're looking for, we become their greatest allies, because who can help them better to meet what students will see as eminently reasonable standards provided they are reasonable standards than us. It's hard for me even to remember the last time that I've had a great complaint from a student. I'm not saying that I don't have students enter my office, having received a grade or been through a peer assessment and saying, you know, I really thought I did a little better than this, but they're not coming in and arguing that the grade is wrong. Instead, they typically come in to open a discussion about how we're conceptualizing that grade and why what am I seeing in it that they didn't see? So for example, a student might come in and say, you know, Perlman, I really thought I had to achieve a stronger grade for analysis. And I could say to them things like, Well, okay, I understand that show me where you see more analysis than what I see, show me what you're identifying as analysis that I am not. And the reason I'm saying I, by the way, is that depending upon the class I'm teaching, it's not necessarily possible for all of the papers to be graded by their peers. And sometimes I'm the one who has to execute on that simply because of time constraints. But they might come in and say, you know, I thought I saw more analysis. And I can say, show me where you thought you saw more demonstrated, in your paper, show me. And most of the time, what happens is something really interesting, the vast majority of the time, I'm going to say nine out of 10 times, maybe eight out of 10 times, the students will withdraw their comment, they'll look at their paper, and then they'll say, oh, you know what, I guess I really can't identify the moments. I guess, I was feeling as though I did more than I actually did on the page. Or sometimes they'll say, you know, what about this? Isn't this an analytic move? And I might say, Well, tell me why you think so. And once in a while, they can demonstrate why it is when maybe I didn't see it as that initially. Most of the time though, it just opens a good natured fair minded discussion about why they're perceiving it to be that way when I'm not. And we work through it as colleagues. But the student coming in and pounding the table and saying that they deserved a higher grade and asking why they didn't get a better grade and telling me all the things that they did and why they deserve that better grade. I haven't encountered that in years. Because when the students understand the standards, and they feel as though those standards are reasonable, I'm not the bad guy anymore in the class, nor need you be. Instead, you can become the ally. The standards are, what the standards are, and the standards are fair, and we understand what those standards are collectively. Now, let me help you get there. Let me work with you as much as we can, so that you are as successful as you want to be in this class, for your own grade and your achievement. But I want to close roughly where I began, because I want to talk about power. Again, I think what we see online and all the contentiousness online, all the vitriol are people who are suddenly given a capacity for power power to judge how or to assess. And they're suddenly in a position that they've never been in before to be able to assess what other people are saying to effectively graded to say that was a good comment, or that was a stupid comment. That was a good tweet or a dumb tweet. But there are people who have never been trained and acculturated, the constructive use of power to the fair minded use of power, all they have experienced is subjugation to power. And so discourse fails, because of course, discourse fails, because we have not trained them through our educational system, how to contend with discourse when they also have power. And more importantly, if we look at our political structure here in the United States, of course, we're going to have people in this country who are inclined to submit to an authoritarian figure. And what that authoritarian figure says is true, because that is in fact, to the letter, if not to the letter grade, what we have habituated them to do for 12 To 16 to 20 years, which is to accept what is said by an authoritarian figure, to accept the judgment of that authoritarian figure, without question without participation, without power. And to view that as positive. A healthy democracy, a healthy intellectual society is contingent upon not just an empowered populace, but a populace that knows how to use power, don't we need to better train our populace and how to use power? And if grades and assessment are, in fact, the locus of power in the academy? Mustn't we bring power onto the table as part of their education and teach them how to exercise and contend with power properly, effectively, responsibly, constructively? That is not one of the things that we must do. That is the foremost thing we need to do. It makes me think of Foucault's characterizations of power, which I don't want to go into great detail about right now. But I think it's very important. Students participate in their own hegemony, they willingly submit to being assessed by someone else by an authority by educators. But Foucault wrote, and I believe I have it correctly, power is exercised only over free subjects, and only insofar as they are free. And let me unpack that very briefly. And maybe there's a future podcast in a footnote Ian analysis of educational power structures Not that I would be even close to the first person who would take on that task. But the living person has no power over the dead person for there's nothing that the living person can do anymore to the person who's already dead. There's no power to be exercised. I mean, there's power over the body, I suppose. But in terms of the actual person who existed, there's no more power to be exercised. And in the same respect, if all we're doing is slapping a grade on a student's paper based on our own idiosyncratic determination, and for which the student has really very little recourse, and has already voluntarily participated in their own hegemony, we really have very little power as well for power is only measured to the extent that it can face resistance. And if the students lack a capacity for any resistance, how much power can I really be said to have? By contrast, perhaps I want to be careful about overstating this, but I really don't think that I am. When I engage in peer assessment with students, when anyone engages in peer assessment with their students, the students actually come to see just how powerful I am. And my power then comes in my intellectual power, in my wisdom in the breadth of my understanding about the subject matter and assessment and writing and critical thinking and teaching. But they can only really appreciate that when they are empowered themselves for again, as Foucault said, power is exercised only over free subjects, and only insofar as they are free. As Walter Lippmann once said, it takes wisdom to understand wisdom. The music means nothing if the audience is deaf. It takes power within the discipline to understand the power of the discipline. If we are authoritarian ly give students a summative grade and they are completely disempowered, then they cannot appreciate how powerful we are within our disciplines. They cannot appreciate our expertise, and they cannot truly understand the value, depth and complexity of the discipline itself. Only when we give students power and wisdom can they truly start to appreciate why what we say has currency, why when we give them a grade or make an assessment, it's not coming from an idiosyncratic place. It's not subjugating them to authoritarianism. It's not a mystery, but it's coming out of a discourse community that values things for certain important reasons that matter to the world. I return of course to Excalibur, where Arthur did not lose power by handing Excalibur to his foe uriens so that he could be knighted. No, he gained power. Arthur handed uriens Excalibur and knelt down in the moat. He acknowledged humbly that he was not a knight yet, and he asked Urian this tonight him so that he could then offer uriens mercy. In that act. What uriens realized was just how powerful Arthur really was. Only the true king, the true son of Arthur, Penn Dragon, who was truly worthy of wielding Excalibur would have the courage to kneel down, disarmed in front of his foe and asked to be knighted. Giving some power to our students shows them our courage. It shows them that our real power comes not from an authoritarian wielding of grades as a weapon, but from our true intellectual capacities, our standing within our fields and our excellent capacities as educators. Isn't that how we really want to be respected? And it seems important to note in conclusion that what Arthur ultimately created was the round table and at the round table while everyone knew that Arthur was King all of the Knights nevertheless had a say they had a way to contribute, they had purpose and they had meaning. And if that's not a valuable cognate for pure assessment, then I don't know what is.
24:19
Swear faith to me. Firstly, I need by the Lord suckers you you right, I'm not gonna know. You like me? Tonight? I count off your mercy What's this? What's this? Hey y'all the name of God St. Michael and St. George, I give you the right to bear arms and the power to me justice. A duty I will solemnly obey as night and King. I never saw this rise, King I am your humble Knight and I swear allegiance to the courage in your veins. so strong it is. Its source must pay do I doubt your noble
